-- Logs begin at Mon 2022-09-12 23:12:54 UTC, end at Tue 2022-09-13 00:15:07 UTC. -- Sep 12 23:37:27.365586 dv-cinder-820027-26-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 12 23:37:27.375428 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 12 23:37:27 2022] *** Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: nodename: dv-cinder-820027-26-iscsi Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: machine: x86_64 Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: clock source: unix Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: pcre jit disabled Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: detected number of CPU cores: 4 Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: current working directory: / Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: detected binary path: /usr/bin/uwsgi-core Sep 12 23:37:27.377389 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: your processes number limit is 47680 Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: your memory page size is 4096 bytes Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: detected max file descriptor number: 2048 Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: lock engine: pthread robust mutexes Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: thunder lock: enabled Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 12 23:37:27.377978 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 12 23:37:27.416512 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: Python main interpreter initialized at 0x557e9435d9c0 Sep 12 23:37:27.416512 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: python threads support enabled Sep 12 23:37:27.416512 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: your server socket listen backlog is limited to 100 connections Sep 12 23:37:27.416512 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: your mercy for graceful operations on workers is 90 seconds Sep 12 23:37:27.416916 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: mapped 403077 bytes (393 KB) for 2 cores Sep 12 23:37:27.416916 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: *** Operational MODE: preforking *** Sep 12 23:37:27.417227 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: *** uWSGI is running in multiple interpreter mode *** Sep 12 23:37:27.417227 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: spawned uWSGI master process (pid: 87831) Sep 12 23:37:27.417455 dv-cinder-820027-26-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 12 23:37:27.418019 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: spawned uWSGI worker 1 (pid: 87846, cores: 1) Sep 12 23:37:27.418350 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: spawned uWSGI worker 2 (pid: 87847, cores: 1) Sep 12 23:37:27.418501 dv-cinder-820027-26-iscsi devstack@keystone.service[87831]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 12 23:37:31.798295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:37:31.798295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: from cryptography.utils import int_from_bytes Sep 12 23:37:31.799177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:37:31.799177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: from cryptography.utils import int_from_bytes Sep 12 23:37:31.848674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 23:37:31.848847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 23:37:31.849071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] command line args: [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 23:37:31.849298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 23:37:31.849473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ================================================================================ {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 23:37:31.849722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] admin_token = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.849971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] config_dir = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.850157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.850337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] config_source = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.850514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] debug = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.850713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_config_append = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.852883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_dir = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.853363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_file = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.853363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.853685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.853783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.853997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.854179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.854379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.854583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.854828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.855062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.855321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.855579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.855781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.856047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.856293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.856560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.857039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.857266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] publish_errors = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.857465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.857649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.857839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.858073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.858274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.858501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] standard_threads = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.858732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.858990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.859131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.859329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] use_journal = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.859513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] use_json = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.859670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] use_stderr = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.859829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] use_syslog = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.860058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:31.860310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.860546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.860801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.861037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.861281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.861556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.861757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.861989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.862232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.862476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.862720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.862950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.863192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.863451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.863696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.863939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.864171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.864422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.864672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.864912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.865145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.865395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.865688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.866182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.866405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.866636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.866850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.867058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.867345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.867560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.867805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.867996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.868264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.868477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.868732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.868945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.869189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.869380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.869641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.869842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.870092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.870295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.870524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.870725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.870973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.871203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.871457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.871734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.871971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.872205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.872449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.872677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.872910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.873143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.873378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.873616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.873836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.874078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.874324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.874557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.874789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.875004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.875289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.875571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.875758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.875965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.876200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.876464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.876698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.877301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.877553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.877789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.878024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.878258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.878504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.878740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.879247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.879474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.879723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.879957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.880188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.880420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.880652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.880885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.881145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.881379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.881623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.881859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.882093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.882330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.882564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.882798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.883029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.883274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.883504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.883732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.883966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.884192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.884429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.884665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.884910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.885144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.885397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.885653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.885893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.886123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.external = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.886351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.886643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.886872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.887098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.password = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.887396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] auth.token = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.887636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.887868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.888111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.888340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.888582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.888825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.889052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.889285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.889530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.889779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.890021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.890261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.890501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.890732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.890974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.891336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.891506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.891757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.892283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.892542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.892781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.893036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.893267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.893503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.893730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.893972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.894198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.894443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.894674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.894924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.895157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.895432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.895667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.895911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.896144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.897629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.898687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.898687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.898687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.898687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.898687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.899752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.899752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.899752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.899752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.900469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.900469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.900469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.900469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.902308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.902308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.902648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.902807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.902991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.903257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.903480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.903703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.903957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.904198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.904382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.904641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.904850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.905104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.905297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.905711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.906237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.906469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.906725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.906917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.907201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.907430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.907769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.907994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.908211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.908382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.908592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.908825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.909066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.909271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.909545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.909777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.910032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.910225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.910482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.910675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.910921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.911150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.911402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.911660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.911902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.912144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.912388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.912614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.912857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.913089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.913320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.913554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.913788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.914029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.914273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.914509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.914759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.914971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.915214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.915476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.915710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.915944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.916188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.916427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.916657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.916894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.917134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.917382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.917627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.917861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.918109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.918357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.918654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.918884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.919119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.919378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.919669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] role.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.919849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] role.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.920072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.920319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.920566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.920810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.921055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.921300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.921544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.921786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.922031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.922267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.922512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.922757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.923002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.923261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.923704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.923939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.924191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.924479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.924679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.924919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.925153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.925403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.925653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.925893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.926148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.926354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.926583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.926829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.927083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.927353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.927591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.927825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.928054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.928289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.928524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.928776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.929004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.929257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.929503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.929744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.929986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.930229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.930473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.930708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.930943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.931174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.931440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.931716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.931943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.932184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.932450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.932709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.connection = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.933322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.933567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.933802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.934040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.934301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.934536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.934778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.934997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.935271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.935491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.935734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.935966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.936233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.936443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.936684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.936912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.937151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.937384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:31.937552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87847) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 23:37:31.969423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:31.969725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:31.969891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:31.978597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.006743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.007096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.061359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.114056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.114264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.278924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.279175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.279397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.279575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.279821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.280032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.304356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.304593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.304801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.305027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.305249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.305462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.305677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.318295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.331245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.331483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.331707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.331926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.332145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.332357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.332568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87847) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.349244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.349554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:37:32.349773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.349964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:37:32.350221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.350485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.350743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.351004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.360723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.361057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.361377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.361761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.368906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.371961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.372245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.372522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.372797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.373075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.373321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.373588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.373858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.374152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.374412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.374679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.397270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.398790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.399048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.403902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.404181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.404459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.410725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.411011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.416895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.417192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.417475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.417760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.418046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.418321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.418619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.418883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.419626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:37:32.419626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: from cryptography.utils import int_from_bytes Sep 12 23:37:32.420180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 12 23:37:32.420180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: from cryptography.utils import int_from_bytes Sep 12 23:37:32.434860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.435162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.439031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.442265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.442558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.446734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.449873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.453058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.453356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.453635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.453913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.454188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.454459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.467617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 12 23:37:32.467759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 12 23:37:32.467986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] command line args: [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 12 23:37:32.468251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 12 23:37:32.468393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ================================================================================ {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 12 23:37:32.468666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] admin_token = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.468913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] config_dir = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.469109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.469262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] config_source = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.469480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] debug = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.469691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.469917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.470037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.470115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.470317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.470411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.470561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.470643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.470789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.470871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.470938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_config_append = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.471134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.471410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_dir = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.471585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_file = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.471820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.472050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.472297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.472542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.472720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.472896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.473138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.473388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.473621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.473851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.474084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.474316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.474560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.474803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.475077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.475601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.475783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] publish_errors = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.476013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.476242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.476432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.476673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.476912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.477139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] standard_threads = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.477369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.477610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.477785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.477957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] use_journal = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.478138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] use_json = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.478309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] use_stderr = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.478481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] use_syslog = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.478709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 12 23:37:32.478971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.479206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.479488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.479725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.479976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.480213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.480461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.480696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.480942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.481218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.481467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.481594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.481710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.481937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.482174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.482421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.482665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.482900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.483154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.483438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.483555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.483666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.483906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.484159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.484420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.484672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.484920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.485151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.485273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.485387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.485603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.485720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.485831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.486073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.486320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.486549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.486780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.487025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.487292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.487531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.487778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.488008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.488242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.488485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.488714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.488827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.488951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.489091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.489207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.489374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.489487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.489610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.489744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.489860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.490059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.490173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.490287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.490513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.490747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.490980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.491244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.491478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.491714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.491952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.492188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.492423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.492656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.492886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.493137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.493377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.493614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.493846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.494105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.494342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.494573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.494809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.495044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.495331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.495571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.495817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.496295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.496529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.496835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.497090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.497327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.497564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.497806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.498041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.498288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.498522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.498755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.498992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.499244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.499483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.499742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.499982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.500229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.500468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.500705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.500945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.501181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.501417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.501650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.501881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.502110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.502340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.502573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.502807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.503046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.503313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.503566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.503737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.503854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.504046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.504160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.504306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.504429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.504542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.504718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.504834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.external = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.504982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.505170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.505287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.505497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.505610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.505723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.password = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.505833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.505953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] auth.token = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.506170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.506405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.506651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.506880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.507126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.507394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.507626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.507861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.508107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.508353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.508601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.508861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.509103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.509353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.509579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.509830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.510067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.510317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.510602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.510853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.511107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.511372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.511631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.511864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.512099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.512368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.512616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.512845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.513093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.513332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.513584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.513818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.514071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.514308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.514556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.514791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.515038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.515314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.515543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.515778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.516026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.516256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.516493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.516742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.516991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.517222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.517452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.517682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.517925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.518171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.518418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.518671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.518917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.519189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.519447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.519684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.519913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.520149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.520379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.520612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.520849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.521086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.521333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.521564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.521809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.522054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.522286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.522530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.522774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.523002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.523259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.523490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.523728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.523963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.524206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.524442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.524677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.524924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.525172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.525402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.525630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.525878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.526123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.526357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.526592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.526950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.527064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.527177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.527439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.527701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.527990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.528245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.528500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.528752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.528992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.529225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.529464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.529701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.529939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.530170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.530418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.530665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.530912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.531159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.531440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.531558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.531656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.531907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.532144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.532382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.532622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.532953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.533805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.533805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.533805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.533805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.534212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.534330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.534490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.534728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.534965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.535213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.535473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.535708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.535944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.536184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.536432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.536679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.536794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.536915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.537150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.537400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.537637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.537883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.538118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.538357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.538591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] role.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.538837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] role.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.538950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.539061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.539346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.539591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.539844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.540093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.540342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.540588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.540836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.541084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.541199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.541311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.541528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.541643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.541790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.542038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.542153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.542285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.542706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.542943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.543195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.543471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.543745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.543995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.544232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.544485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.544738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.544979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.545212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.545444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.545678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.545928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.546187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.546437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.546677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.546913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.547146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.547409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.547647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.547894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.548131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.548389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.548637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.548877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.549123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.549368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.549618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.549855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.550094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.550330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.550578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.550859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.551090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.551355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.551625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.551887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.connection = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.552001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.552121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.552370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.552607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.552849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.553086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.553329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.553567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.553805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.554051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.554164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.554275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.554521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.554756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.555004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.555261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.555511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.555744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.555989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.556227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 12 23:37:32.556399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87846) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 12 23:37:32.558844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.559472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.559873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.560296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.569198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.569422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.569702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.576578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87847) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.576881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.577158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.577434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.577716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.577990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.578260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.578530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.578784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.579061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.579400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.579688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.580025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.580311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87847) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.582183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.582415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.582643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.591306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.609772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.609978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.614414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557e9435d9c0 pid: 87847 (default app) Sep 12 23:37:32.617929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebb93-a9a1-4238-9851-11505447cee1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:32.618209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebb93-a9a1-4238-9851-11505447cee1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:32.618476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebb93-a9a1-4238-9851-11505447cee1 None None] PATH_INFO: `/v3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:32.622784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Sep 12 23:37:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:32.648818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.685026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.685312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.828815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.829012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.829183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.829406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.829628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.829844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.843747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.843973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.844198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.844421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.844640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.844856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.845071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.853843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.857397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.857627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.857850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.858070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.858284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.858497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.858751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87846) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 12 23:37:32.869393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.869641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:37:32.869947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.870156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 12 23:37:32.870433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.870707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.870920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.871256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.881025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.881327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.881594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.881874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.889114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.892066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.892355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.892643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.892944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.893239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.893499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.893772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.894046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.894318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.894587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.894867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.917565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.919113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.919421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.924009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.924268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.924571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.931029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.931335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.935872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.936167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.936449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.936742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.937025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.939201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.939516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.939811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.955665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.955882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.959727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.962851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.963147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.967746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.970834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.974130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.974433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.974719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.975112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.975427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.975720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.991596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:32.991891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.992161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.992424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:32.999253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.000931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.002448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.002731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.005400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.005701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.005981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.006275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.006565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.017867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.018159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.018442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.018723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.019005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.019295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.019574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.035037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.038114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.041026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.042449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.043985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.044273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.044574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.051044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.052550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.055432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.055734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.056019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.056306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.064744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.065045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.065322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.072298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87846) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 12 23:37:33.072596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.072890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.073170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.073440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.073719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.073995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.074262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.074533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.074840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.075141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.075426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.075726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.075981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87846) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 12 23:37:33.110228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557e9435d9c0 pid: 87846 (default app) Sep 12 23:37:39.471519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:39.472086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:39.472086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:39.473313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:39.481559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:39.482158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:39.482158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:39.552641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: INFO dbcounter [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] Registered counter for database keystone Sep 12 23:37:39.572795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG oslo_db.sqlalchemy.engines [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87846) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 23:37:39.580401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writer thread running {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 23:37:39.769237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG passlib.handlers.bcrypt [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87846) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 23:37:39.769535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG passlib.handlers.bcrypt [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87846) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 23:37:39.828146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:39.829803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:39.968019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1/3] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 23:37:40.417094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:40.487581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: INFO dbcounter [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] Registered counter for database keystone Sep 12 23:37:40.513887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG oslo_db.sqlalchemy.engines [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87847) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 12 23:37:40.520883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writer thread running {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 12 23:37:40.640147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311a59f-57e5-4b44-9d58-69210bde8294 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:40.644926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:40.645573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:40.645848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:40.646112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:40.654734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:40.706487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING keystone.common.rbac_enforcer.enforcer [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 23:37:40.706755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dd185-3f41-43f5-921b-31e1c9a39476 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:40.717025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 3/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Sep 12 23:37:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:37:40.723318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:40.741019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711264aa-2ab5-4f7a-b0fd-717762052ee1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:40.746036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:40.747405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:40.747687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:40.747955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] PATH_INFO: `/v3/projects/16141215d11e48819148a2d135301f22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:40.749469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] RBAC: Authorizing `identity:get_project(project_id=16141215d11e48819148a2d135301f22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:40.801518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 12 23:37:40.801843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b5e4c-ddea-457b-85e3-91a0446a17de admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:40.806700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 2/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Sep 12 23:37:40 2022] GET /identity/v3/projects/16141215d11e48819148a2d135301f22 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:42.927925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a0f48-ed27-4372-97a9-190a5f6a6f96 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:42.928503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a0f48-ed27-4372-97a9-190a5f6a6f96 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:42.928503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a0f48-ed27-4372-97a9-190a5f6a6f96 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:42.932966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 4/6] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:42.938835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:42.939311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:42.939399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:42.960341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:42.961896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:42.970821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 3/7] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 12 23:37:43.237545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a0f48-ed27-4372-97a9-190a5f6a6f96 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:43.251955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a0f48-ed27-4372-97a9-190a5f6a6f96 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:43.257050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:43.257660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:43.257935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:43.258208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:43.266602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:43.268403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1099b77-5308-4657-b96e-c7aee4b8c361 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:43.277557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 5/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Sep 12 23:37:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:37:43.283428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:43.293052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4b3d2-dc40-411c-80f9-035dd03b3ee4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:43.299389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:43.300037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:43.300319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:43.300584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:43.302063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:43.303963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9c0ce-a83f-406a-8ef0-7002b4329dc0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:43.305531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 4/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Sep 12 23:37:43 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:45.518768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84339-56f3-41dd-9b6c-ff97d4867ca2 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:45.519313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84339-56f3-41dd-9b6c-ff97d4867ca2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:45.519313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a84339-56f3-41dd-9b6c-ff97d4867ca2 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:45.520606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:45.526262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91246aa-c0f1-4d3d-aaa6-e49d9624dd8e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:45.526711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91246aa-c0f1-4d3d-aaa6-e49d9624dd8e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:45.526893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91246aa-c0f1-4d3d-aaa6-e49d9624dd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:45.545342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f91246aa-c0f1-4d3d-aaa6-e49d9624dd8e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:45.546563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f91246aa-c0f1-4d3d-aaa6-e49d9624dd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:45.638728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 5/11] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:45.688374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65d177-efc1-4459-96f5-a116c01b4e93 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:45.688652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65d177-efc1-4459-96f5-a116c01b4e93 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:45.688917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65d177-efc1-4459-96f5-a116c01b4e93 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:45.690269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 7/12] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:45.695763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:45.696060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:45.696336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:45.717456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:45.718673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:45.735940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:45.790395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 6/13] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:45.902676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65d177-efc1-4459-96f5-a116c01b4e93 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.053845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.098983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65d177-efc1-4459-96f5-a116c01b4e93 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.134659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.136168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.136168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.136168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.148736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:37:46.159627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:37:46.160585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:37:46.161366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 8/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:37:46.162011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d5066-897b-4416-b91f-f0eed3ca59d9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.166914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.200410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.201045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.201322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.201594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.204073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.223713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.272482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.273957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.286764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f099ca9-b855-4b09-b1ab-4e4a2ec4bcc1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.320989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.321612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.321886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.322151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.322813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:46.323108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.324599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.337205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 9/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:46.343210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.378926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 7/16] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:37:46.385371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.447156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.448004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.455335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ab194-372d-4541-99ba-bfab46effb32 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.491581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.492225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.492529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.492802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.494422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.495913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.497572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 10/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:46.499564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bded6e-a1ff-422b-bc73-963803ddc752 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.503428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.535745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.536673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.536959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.538353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.550288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.552097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:37:46.572675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:37:46.573595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:37:46.574386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:37:46.574386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:37:46.574386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:37:46.574386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 8/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:37:46.579951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.617459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8e719-6a5c-4003-87b6-a1e10c85e4c9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.659586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.660348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.660645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.660897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.662203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.663716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.665391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 11/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:46.672370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:46.694999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faafba-c940-4291-a8ad-753886d7d101 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.697574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.712973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:46.734725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.735393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.735684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.736061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.737177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:46.737669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.740026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e67e65-a600-4a7f-8da1-2ba300c00e3d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.767411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 9/20] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:46.811931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bfc5f-f27d-4f5b-81e7-6b3a801c622f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:46.855828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:46.856387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:46.856663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:46.856928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:46.857693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.domain.id=default, user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:46.857995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:46.859521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df1c5-faab-4cde-bcc1-82e78a13afa7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:46.870087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 12/21] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Sep 12 23:37:46 2022] GET /identity/v3/role_assignments?user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712&scope.domain.id=default&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 233 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:49.730110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6ff9a-5e08-45bc-887c-47c5ea4ec02f None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:49.731285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6ff9a-5e08-45bc-887c-47c5ea4ec02f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:49.731768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6ff9a-5e08-45bc-887c-47c5ea4ec02f None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:49.733488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 10/22] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:49.739764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:49.740732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:49.741240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:49.765313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG passlib.handlers.bcrypt [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87847) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 12 23:37:49.766035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG passlib.handlers.bcrypt [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87847) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 12 23:37:49.832447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:49.834552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f777bed2-6421-4750-8042-96b10e00a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:49.937103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 13/23] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:50.008092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd85bc-3348-44e9-83a7-21f4d11550bd None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.008680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd85bc-3348-44e9-83a7-21f4d11550bd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.009134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd85bc-3348-44e9-83a7-21f4d11550bd None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.010781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 11/24] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:50.013933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0de9-247b-44ca-9c32-724419f12866 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.014478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0de9-247b-44ca-9c32-724419f12866 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.014931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa0de9-247b-44ca-9c32-724419f12866 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.017552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c201-471b-4bfc-9183-ef944706f2d4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.018067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c201-471b-4bfc-9183-ef944706f2d4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.018542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c201-471b-4bfc-9183-ef944706f2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.019511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 14/25] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:50.030308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.032837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.034279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.053683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-24d4c201-471b-4bfc-9183-ef944706f2d4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:50.061435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-24d4c201-471b-4bfc-9183-ef944706f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:50.075753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:50.079252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:50.131712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:50.194282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 12/26] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 12 23:37:50.216518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838fc7d-64c7-49ac-b33e-8c42f37fcbc9 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.216779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838fc7d-64c7-49ac-b33e-8c42f37fcbc9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.217058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838fc7d-64c7-49ac-b33e-8c42f37fcbc9 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.218708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:37:50.231461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 15/28] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:50.245307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.246013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.246505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.276870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:50.278512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:50.378153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 14/29] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:50.573281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:50.606050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:50.652394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:50.771921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab139cc-a46d-4192-85a0-fa05f95e2c19 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:50.827149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:50.828101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.828552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.829288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.830156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:50.832264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:50.841478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03526e-2c07-4335-aae1-4fcb03407b48 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:50.919321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:50.920576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:50.921047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:50.921532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:50.954357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:50.957134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:37:50.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:37:50.986418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:37:50.987200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:37:50.987200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:37:50.987200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:37:50.988047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 15/30] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:37:50.989603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.007908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 16/31] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Sep 12 23:37:50 2022] POST /identity/v3/roles => generated 225 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:37:51.009270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.081568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:51.228033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:51.259378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d4da5-3393-476e-bd39-5412ff54a9f8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.265317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d87-deb6-4a89-beb3-b42984c458a8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.306596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:51.307957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.308501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.308930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.309798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:51.311533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:51.316709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:51.318022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.318615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.319256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.344019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:37:51.361146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:37:51.362061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:51.362800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:37:51.365241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 17/32] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:37:51.367408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.468336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 16/33] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Sep 12 23:37:50 2022] POST /identity/v3/roles => generated 219 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:37:51.470184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.511105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:51.561548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdecab-22fa-4a1d-a6d1-6b3c9d96de1e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.616424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7562f5e-019d-469b-ac6c-5b3860e4cbf2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.627253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:51.628424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.628929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.629422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.630626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:51.631159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:51.634172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87f08-5679-4793-ab39-da91d754871e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:51.656791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:51.657436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.657713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.657977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.658668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:51.658965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:51.660608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273b3af-1ef6-4a53-b228-128b9436fd76 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:51.664626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:51.681056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 18/34] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:51.681056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98aa3-a872-4d8a-a7b8-1c0b0de30490 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.681056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98aa3-a872-4d8a-a7b8-1c0b0de30490 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.681056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe98aa3-a872-4d8a-a7b8-1c0b0de30490 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 17/35] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a019-56a2-4c58-a423-73d394501c3a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a019-56a2-4c58-a423-73d394501c3a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a019-56a2-4c58-a423-73d394501c3a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 18/36] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:51.681474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236a019-56a2-4c58-a423-73d394501c3a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.683315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 19/37] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:51.685038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.685283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.685630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.707095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:51.708772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:51.799557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236a019-56a2-4c58-a423-73d394501c3a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.821141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 20/38] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:51.822783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:51.922696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:51.968862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34181c2a-55c2-4e6f-a0c4-dc539bb521d2 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:51.969092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:51.969703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:51.969869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:51.970148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:51.970817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:51.972474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bffb9-5eef-4dcc-9836-2cd1c2d7abc8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.006882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:37:51 2022] POST /identity/v3/projects => generated 301 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:37:52.008223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.008514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.008789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.032808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:52.034675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:52.040975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.042005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.042512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.043038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.045927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.048473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.051487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 21/40] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:37:51 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:52.060980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:52.122814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:52.159030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 20/41] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:52.212226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:52.212991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92d10c-fc71-4076-bfd2-ddf40b91c4a6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:52.255209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.256014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.256384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.256722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.258035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.259668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.261867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 22/42] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:52.273215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:52.278739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:52.347631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689f874-ee80-480a-afdd-bd806896ac0e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:52.405631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.406612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.407055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.407532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.408597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.410583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.439332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 21/43] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Sep 12 23:37:52 2022] POST /identity/v3/roles => generated 223 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:37:52.452923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7901-4df7-42cf-8050-2aa80f9a7e80 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:52.505970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.506773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.507263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.507666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] PATH_INFO: `/v3/domains/default/users/f9f809fdc6b04f0ca4d4c2bc46e0a712/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.510000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.511949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.530845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:52.535416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 23/44] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Sep 12 23:37:52 2022] PUT /identity/v3/domains/default/users/f9f809fdc6b04f0ca4d4c2bc46e0a712/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:37:52.611623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:52.706960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07baab-f4e8-4844-b132-5593c9d6c45d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:52.751514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.752188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.752463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.752731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.753982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.755549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc60d46-59fe-4ba1-9e8a-318f992fef29 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.757318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:52.763101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:52.833705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:52.878748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96345c46-8020-43a3-980b-224c2bb0d26e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:52.919632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:52.920262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:52.920541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:52.921108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:52.921791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:52.923301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054ffde-d541-41d2-929d-e41f3e54d883 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:52.962033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:37:52 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:37:54.915497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe16a-ad87-4f6f-b213-ae076b667b77 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:54.916070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe16a-ad87-4f6f-b213-ae076b667b77 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:54.916070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe16a-ad87-4f6f-b213-ae076b667b77 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:54.917436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:54.923157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826be68-74cc-447d-a711-36df575f8e50 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:54.923638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826be68-74cc-447d-a711-36df575f8e50 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:54.923722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826be68-74cc-447d-a711-36df575f8e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:54.943747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f826be68-74cc-447d-a711-36df575f8e50 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:54.944972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f826be68-74cc-447d-a711-36df575f8e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:55.014413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:55.243056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8d8d8-6e00-4daa-a00a-2f96660a398a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.243283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8d8d8-6e00-4daa-a00a-2f96660a398a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.243568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8d8d8-6e00-4daa-a00a-2f96660a398a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.245229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 24/49] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:55.250791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.251179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.251391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.271133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:55.272393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:55.273912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8d8d8-6e00-4daa-a00a-2f96660a398a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.314835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:55.345201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 26/50] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:55.387204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8d8d8-6e00-4daa-a00a-2f96660a398a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.397018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:55.429480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:55.430116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.430368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.430657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.443288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:37:55.449696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:37:55.450685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:37:55.451686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 25/51] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:37:55.456967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.568421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931639d8-a30a-4d75-be4c-df20483e4965 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.603491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:55.604114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.604400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.604673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.605406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:55.605709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:55.607119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:55.612091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.616127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 27/52] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:37:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:55.621920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.663656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:55.722687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc7f8-e67b-47f7-8cfe-5758d6919c7b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.763958 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:55.764623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.764907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.765186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.766400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:55.767960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:55.769652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:55.775759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.814811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:55.845275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bf6e-955b-445c-89ea-7bb5dbf52f0b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.878817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:55.879483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.879758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.880031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.881167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:55.882616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:55.882928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463057f-2234-4898-99a3-ac7dead28914 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.884240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 28/54] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:37:55 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:55.890131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:55.918934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:55.919585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:55.919869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:55.920141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:55.920794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:55.922270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:55.949905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 27/55] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Sep 12 23:37:55 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:37:55.997118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a535546-4081-4487-a2ac-85a31f8a9a0c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:55.999748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:56.038849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:56.039531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:56.039795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:56.040075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:56.041332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:56.042876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2d00c-3294-4858-a205-c5b94c275dd2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:56.044607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 29/56] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:37:56.051109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:56.160664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:56.178774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307c45-f645-4421-8454-0cd74bbbd1c9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:56.215974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:56.216595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:56.216873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:56.217145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:56.217900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.domain.id=default, user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:37:56.218200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:56.219723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427e00-65cc-4a44-95a1-65f1260da983 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:56.227648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Sep 12 23:37:56 2022] GET /identity/v3/role_assignments?user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712&scope.domain.id=default&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 530 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:58.150839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597367e5-62f3-4968-b6be-a73707fa1fe3 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:58.151402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597367e5-62f3-4968-b6be-a73707fa1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:58.151402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597367e5-62f3-4968-b6be-a73707fa1fe3 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:58.152824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 30/58] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:37:58.158268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:58.158694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:58.158775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:58.178378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:37:58.179624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:37:58.251001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:37:58.520029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597367e5-62f3-4968-b6be-a73707fa1fe3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:58.628461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597367e5-62f3-4968-b6be-a73707fa1fe3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:58.664674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:58.665314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:58.665629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:58.665900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:58.667089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:58.668580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a02e5-ed49-42cd-9d83-f9fddfd09732 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:58.670173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 31/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:37:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:37:58.676156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:37:58.761764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:37:58.782471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3319b4-a251-4aaa-bb81-11d18e7f71ea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:37:58.817903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:37:58.818514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:37:58.818792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:37:58.819059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:37:58.819734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:37:58.821198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef2a57-9c4b-418c-b115-93ce56ec68db None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:37:58.848491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 30/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:37:58 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:02.004136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0297e-b0b7-487d-8d3a-05fb13a1c7d1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.004704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0297e-b0b7-487d-8d3a-05fb13a1c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.004704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0297e-b0b7-487d-8d3a-05fb13a1c7d1 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.006044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 32/62] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:02.013113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1c9-0671-4f24-9c48-51b3c1563421 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.013804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1c9-0671-4f24-9c48-51b3c1563421 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.014297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1c9-0671-4f24-9c48-51b3c1563421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.055913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-455eb1c9-0671-4f24-9c48-51b3c1563421 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:02.058397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-455eb1c9-0671-4f24-9c48-51b3c1563421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:02.189808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 31/63] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:02.527837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0297e-b0b7-487d-8d3a-05fb13a1c7d1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:02.564653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9016-8a01-4993-b959-9e5410345cc3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.566530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9016-8a01-4993-b959-9e5410345cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.567212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b9016-8a01-4993-b959-9e5410345cc3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.570038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 32/64] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:02.585016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf77c4-41c8-497e-8155-5f6020991fb9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.586689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf77c4-41c8-497e-8155-5f6020991fb9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.587460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf77c4-41c8-497e-8155-5f6020991fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.629652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-fedf77c4-41c8-497e-8155-5f6020991fb9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:02.631992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fedf77c4-41c8-497e-8155-5f6020991fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:02.643658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:02.737072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0297e-b0b7-487d-8d3a-05fb13a1c7d1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:02.750971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:02.762158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 33/65] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:02.763531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dcd07-3109-440c-a45a-a67829f1c2f1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.763848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dcd07-3109-440c-a45a-a67829f1c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.764128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dcd07-3109-440c-a45a-a67829f1c2f1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.765445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 34/66] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:02.772487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.772487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.772487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.795420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:02.796621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:02.799667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:02.801278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:02.801557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:02.801830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:02.830107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:02.835431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:02.837107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:02.839554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:02.843908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 33/67] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:02.847250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:02.902329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 35/68] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:03.001340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.005142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6db71-600b-4d9b-aeac-ef184e6d6cf5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.042083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.042751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.043043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.043352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.044039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:03.044341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:03.045813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:03.055106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:03.061658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.205161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bd8f4-550e-4a00-84d5-1953a1d897a7 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.208762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.259482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.274749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.275431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.275698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.275956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.307447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:03.309316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:03.311519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:03.318598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.405791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0f89-6130-4fa8-b032-afb5d5171f55 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.446994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fbdb2-2c83-4560-8922-31598a2a9780 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.478756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.479980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.480475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.481007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.491274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.492106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.492382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.492740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.503951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:03.516148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:03.517056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:03.517791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:03.518858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:03.520671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.554295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.565230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.572603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:03.574226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fbede-bd68-4e50-bd18-e08acf3943a3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:03.671914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efb677-94ca-4962-86eb-f19255a2ce96 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.714814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 37/72] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:03.715041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.715138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6a1-3aad-4987-9bce-a65928fcda92 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.715206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.715206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.715375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6a1-3aad-4987-9bce-a65928fcda92 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.715443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.715506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6a1-3aad-4987-9bce-a65928fcda92 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.720357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 38/73] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:03.720357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6c6a1-3aad-4987-9bce-a65928fcda92 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.732881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:03.739505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:03.740474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:03.741216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 36/74] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:03.742655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.839453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.859587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6c6a1-3aad-4987-9bce-a65928fcda92 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.867629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:03.892372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8a0c-5b68-4cf9-83b4-72deb72e0f3f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:03.907251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.908297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.908783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.909250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.910451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:03.911001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:03.913612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904800a-ddb3-4e79-9773-7127132141aa None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:03.928041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 39/75] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:03.929863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:03.930239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0380dae-8869-458c-af2b-5e2aafa39bb7 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.930502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:03.930706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0380dae-8869-458c-af2b-5e2aafa39bb7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.930839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:03.931043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.931177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0380dae-8869-458c-af2b-5e2aafa39bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:03.932715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:03.934252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:03.936149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 37/76] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:03.936996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:03.965444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a0380dae-8869-458c-af2b-5e2aafa39bb7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:03.967496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a0380dae-8869-458c-af2b-5e2aafa39bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:04.066477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f10b7-db58-45b1-81e2-a438e62cd23f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.069030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 40/77] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:04.070549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39d55-78dc-497e-8387-d158e0dfd0f8 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.071053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39d55-78dc-497e-8387-d158e0dfd0f8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.071565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a39d55-78dc-497e-8387-d158e0dfd0f8 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.074257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 41/78] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:04.075548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a39d55-78dc-497e-8387-d158e0dfd0f8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.114118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.115530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.115989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.116424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.117517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:04.118022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:04.120740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:04.134741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 38/79] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:04.136537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.161048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:04.209871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:04.241280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a39d55-78dc-497e-8387-d158e0dfd0f8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.290775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6e8e-f046-4ee1-9883-29ada12344e6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.300277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.301348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.301833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.302420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.304296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:04.307397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62bdec-db5e-4769-83b3-9be8cb66691d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:04.310431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 42/80] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:04.312605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.313086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.313551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.343623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.344255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.344535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.344806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.345119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:04.345544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:04.345850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:04.347105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:04.347343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:04.354339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 39/81] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=cc467837f6074b9391a11b0e57406d12 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:04.355283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.477156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:04.485097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 43/82] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:04.487094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.536508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f2325-3eff-4596-8808-7c61fc5edce7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.607787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.610033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.610317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.610604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.612045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:04.613545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:04.615248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 40/83] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:04.616068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.660545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:04.722169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16148b9-d6e4-4552-b0d5-fc75fe4701cf None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.740097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd45e89-5b58-45b7-8242-a75f42793c6f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:04.785945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.786789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.787176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.787588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.802901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:04.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:04.812706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:04.814437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:04.817261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 41/84] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:04.818166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:04.818964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:04.819649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:04.820282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:04.820976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:04.826920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:04.968386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:04.971797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.000911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473ad4-cad5-4695-8fdb-099de146c96d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.054628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.055665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.055944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.056220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.111039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.144756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.146249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.150231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.158637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 44/85] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:05.158637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.273147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13242c99-d1b4-4f4d-ae04-b94d366ffb08 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.306278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 42/86] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:05.307369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.310843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.312152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.312580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.313005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.314178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:05.314929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.318544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.332923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 45/87] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.334208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.403024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.452924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4498-ae98-4e1b-a461-c3600b40bce0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.457098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.480304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2581b-63b9-46a4-abf0-54080cd54b08 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.513396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.515069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.515536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.516298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.518129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.520459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.523623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 43/88] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.525207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.540908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.541564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.541898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.542276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.544026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.546431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.549132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 46/89] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.550243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.690840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8903-d6c4-4ae1-a0b9-72b5ae41a7aa None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.693043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02942005-ab2c-4112-aa66-118a4cf80a8a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.707569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.732146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.733083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.733507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.733917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.735292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.736962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.738813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 44/90] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.739754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.755945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.756569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.756851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.757140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.758351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.759817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.768110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 47/91] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.768110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.803805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:05.884906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a458c57-a772-406b-aa1a-c985ffcab432 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.886281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8410db6-9f40-45fa-8a0d-bc4cf78fee24 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:05.924453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.925171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.925459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.925800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.926437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.927937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.947820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:05.949009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:05.949489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:05.949901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:05.950869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:05.951212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:05.953335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:05.959332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 45/92] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:05 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:05.960421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:05.960658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 48/93] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/role_assignments?user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:05.961674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:06.079975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:06.189469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b067613-cc5a-4bcc-9336-9902288b39ae None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:06.207377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:06.217238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac63e5-a765-4c72-b8cc-88e2df1400f7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:06.246091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:06.246771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:06.247439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:06.247721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:06.289600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:06.290245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:06.290868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:06.291142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:06.293398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:06.293918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:06.296025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:06.305696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 46/94] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:06.342510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:06.344526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:06.398752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:06.476978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 49/95] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:06.483107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:06.602596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a4442-a7ba-4fab-aa2f-77643103ff0e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:06.647377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:06.648069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:06.648361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:06.648634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] PATH_INFO: `/v3/projects/afb395674b73408f93f3a0fa436e2036` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:06.652541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] RBAC: Authorizing `identity:get_project(project_id=afb395674b73408f93f3a0fa436e2036)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:06.654927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4f9f8-e4ac-4205-a811-f1e7fa53cfaa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:06.657368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 47/96] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:06 2022] GET /identity/v3/projects/afb395674b73408f93f3a0fa436e2036 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:06.664096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:06.804365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da8a68-b334-4183-9f4b-817351518674 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:06.850625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:06.852091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:06.852867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:06.853537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:06.854866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=afb395674b73408f93f3a0fa436e2036, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:06.855615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:06.858294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9964e6-4303-4ee0-80a5-1ab549a43413 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:06.874473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 50/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:06 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=afb395674b73408f93f3a0fa436e2036&role.id=cc467837f6074b9391a11b0e57406d12 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:07.437703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355f1d-94a7-4d45-8b39-c71e9e133f32 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:07.438229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355f1d-94a7-4d45-8b39-c71e9e133f32 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:07.438694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11355f1d-94a7-4d45-8b39-c71e9e133f32 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:07.441330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 48/98] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:07.447687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:07.448272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:07.448272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:07.458331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:07.477351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:07.478721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:07.567008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 51/99] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 12 23:38:08.395718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11355f1d-94a7-4d45-8b39-c71e9e133f32 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:08.460084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:08.615619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11355f1d-94a7-4d45-8b39-c71e9e133f32 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:08.672763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:08.673341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:08.673627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:08.673908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:08.695720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:08.700985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:08.702627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:08.704119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 49/100] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:08.715358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:08.863487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d03e1c-e1af-4b8d-be78-9842084363c6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:08.906258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:08.911744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:08.912399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:08.912714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:08.912937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:08.913664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:08.913971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:08.915481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:08.928479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 52/101] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:08.938115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:09.094948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:09.120447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e9f3f-7169-456c-8c1d-83d577cc6ca9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:09.210024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:09.211617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:09.212373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:09.213151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:09.218476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:09.221621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f5cdc-b559-4edf-b6fb-695b132a0b36 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:09.225024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 50/102] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:08 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:09.235394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:09.381196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bd689-c77a-43aa-b46a-13c35e0e0bce None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:09.382148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bd689-c77a-43aa-b46a-13c35e0e0bce None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:09.382911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bd689-c77a-43aa-b46a-13c35e0e0bce None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:09.389956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 51/103] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:09.397308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:09.398215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:09.399282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:09.452008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:09.454409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:09.547696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c7de6-c1c1-4819-8a93-3acf8fa3c146 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:09.652200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 52/104] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:09.665034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:09.666095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:09.666593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:09.667159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:09.672456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:09.675212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:09.678353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 53/105] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:09 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:09.688786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:09.871274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:09.908386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39445aa9-8d49-494b-91d5-e6b0d871bd8c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:09.981569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:09.982736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:09.983244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:09.983738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/1e961cef8d2643c480253e713a38bdde/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:09.987136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, user_id=1e961cef8d2643c480253e713a38bdde, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:09.991918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:10.027339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 53/106] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:09 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/1e961cef8d2643c480253e713a38bdde/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:10.092413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:10.222808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:10.277703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d6b7-1d82-475e-81c7-d8ffdb415218 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:10.331624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:10.332318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:10.332602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:10.332871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:10.352485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:10.358919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:10.362473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:10.365548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 54/107] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:10.371622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:10.574235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a6e54-d794-46d1-b155-ccf4d958d2a2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:10.622214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:10.622859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:10.623134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:10.623511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:10.624221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:10.624525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:10.626181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2738d-8928-44b0-a9fb-f37f66610483 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:10.639953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 54/108] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:10.648681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:10.949007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3cf2e-3c32-4d3f-9136-ae285702ee3c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:11.020586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:11.062268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b02-201a-4425-b6ef-38058d9e12ab None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.063710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b02-201a-4425-b6ef-38058d9e12ab None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.064304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1b02-201a-4425-b6ef-38058d9e12ab None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.066704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 55/109] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:11.068600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:11.069831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.070454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.071038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.081311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a162d8-be56-46ba-9f87-41845751dfe9 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.081778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a162d8-be56-46ba-9f87-41845751dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.082251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a162d8-be56-46ba-9f87-41845751dfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.088535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:11.093173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:11.096449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 55/110] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:10 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:11.113316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:11.147687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-94a162d8-be56-46ba-9f87-41845751dfe9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:11.152994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-94a162d8-be56-46ba-9f87-41845751dfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:11.159929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:11.417979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 56/111] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:11.532363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608add-b0d6-4137-8622-739af835f113 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:11.583793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e64a29-3b9f-4935-bdc5-e9e02ab60dfe None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.584220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e64a29-3b9f-4935-bdc5-e9e02ab60dfe None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.584367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e64a29-3b9f-4935-bdc5-e9e02ab60dfe None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.586031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 57/112] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:11.597823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609c0c-ada9-40b1-8980-fb41ad2d6bf3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.598419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609c0c-ada9-40b1-8980-fb41ad2d6bf3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.598885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85609c0c-ada9-40b1-8980-fb41ad2d6bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.600161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:11.601214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.601701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.602185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.610969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:11.610969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:11.612078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 56/113] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:11 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:11.619423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:11.655663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-85609c0c-ada9-40b1-8980-fb41ad2d6bf3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:11.657695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-85609c0c-ada9-40b1-8980-fb41ad2d6bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:11.782115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:11.919127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 58/114] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:11.943577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8be3-e4d8-483a-bd93-324ba449e7cd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:11.978751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c2615-96a3-4fb3-b7ed-a2fd538e3c83 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.979659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c2615-96a3-4fb3-b7ed-a2fd538e3c83 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.980341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c2615-96a3-4fb3-b7ed-a2fd538e3c83 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:11.983332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 59/115] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:11.989122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68953902-1ebe-4884-8afe-c023d70420bc None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:11.989834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68953902-1ebe-4884-8afe-c023d70420bc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:11.990701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68953902-1ebe-4884-8afe-c023d70420bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.079268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-68953902-1ebe-4884-8afe-c023d70420bc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:12.079268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-68953902-1ebe-4884-8afe-c023d70420bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:12.102283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:12.103649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:12.104156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:12.104700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/f9f809fdc6b04f0ca4d4c2bc46e0a712/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.108666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:12.112111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:12.155697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 57/116] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:11 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/f9f809fdc6b04f0ca4d4c2bc46e0a712/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:12.157395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:12.309140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:12.322950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 60/117] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:12.344811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb4766-8207-4f77-ab2f-c2ea6965b6ff None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:12.345332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb4766-8207-4f77-ab2f-c2ea6965b6ff None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:12.345801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb4766-8207-4f77-ab2f-c2ea6965b6ff None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.348718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 61/118] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:12.350174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb4766-8207-4f77-ab2f-c2ea6965b6ff None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:12.485938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f176-2bb0-48d8-9c29-fad64169deb0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:12.528220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:12.554662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:12.555779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:12.556634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:12.557126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.570618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-3b29328a-a80e-4275-b162-dd3da74510f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:12.576761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:12.577655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:12.578401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:12.581189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 58/119] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:12.582877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:12.587704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:12.588294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.594874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb4766-8207-4f77-ab2f-c2ea6965b6ff None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:12.611354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:12.613029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:12.682084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:12.683318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:12.683836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:12.684345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:12.709369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:12.721011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:12.722694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 62/120] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:12.724702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:12.749018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 59/121] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:12.750018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:12.856698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:12.980295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e157d-03f6-4e84-81e4-94afea2ff428 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.035557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4bd8b-0d21-4f57-b901-8ebc9bb4b8ba None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.054214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:13.096042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.096839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.096973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.097297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.097926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:13.098232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.099762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.105581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.107165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.107674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.109712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.112476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.115532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 60/122] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:13.117306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.122233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 63/123] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:13.123101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.383280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de697e52-25fd-4af2-b432-963a9e9459d6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.405113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c69b2-648c-4e99-b855-015ab91322cd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.446752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:13.458690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.459768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.460224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.460700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.462371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:13.462879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.466616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.478276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.480048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.480949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.481688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.483121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.484494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 61/124] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:13.486149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.486896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.548329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 64/125] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:38:13 2022] POST /identity/v3/projects => generated 302 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:13.550454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.581173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:13.707940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe324b2-2a63-4297-8316-4d7a6abf4e5a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.777210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.778317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.778711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208743a-15ac-4f7d-86f1-a4617c722d31 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:13.778878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.779300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.781869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:13.784867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:13.788571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 62/126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:13.794750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.875989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:13.880711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:13.881687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:13.882430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:13.928320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:13.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:13.939725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:13.941237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 65/127] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:13.942435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:13.999160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065450a-3b5d-4d6e-997f-9d113a43b46f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:14.009966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:14.014520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:14.088622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:14.089714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.090154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.090554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] PATH_INFO: `/v3/users/886d48441c5548498fa0b04224f8e56b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.116455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] RBAC: Authorizing `identity:get_user(user_id=886d48441c5548498fa0b04224f8e56b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:14.119136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333c726-0f47-4df2-ab84-faead2914d7e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:14.122612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 63/128] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/users/886d48441c5548498fa0b04224f8e56b => generated 289 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:14.125090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649884f-0aea-40b2-987e-7c5c118ccf93 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.125621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649884f-0aea-40b2-987e-7c5c118ccf93 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.126101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649884f-0aea-40b2-987e-7c5c118ccf93 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.129311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 64/129] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:14.130739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649884f-0aea-40b2-987e-7c5c118ccf93 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:14.166612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b18b4-b3ae-406c-9c5d-66b07b600f7d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:14.242653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:14.243893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.244777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.245363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.247716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:14.251027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:14.254520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 66/130] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:13 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:14.260311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:14.374994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649884f-0aea-40b2-987e-7c5c118ccf93 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:14.445075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:14.446169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.446655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.447148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.448459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:14.449172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:14.452143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80600-8636-444c-9c88-3b9790ef5d5f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:14.468519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:14.470933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 65/131] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:14.475516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75b968-2092-4aeb-a925-7e69ec12cf85 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:14.482018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.482500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.482980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.516745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:14.518004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:14.542705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:14.546593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:14.548289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.549010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.549744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.609182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 66/132] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:14.610160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:14.728926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:14.732370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:14.752795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c15b1-032a-4ea2-a889-3d200b279df0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:14.801490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:14.802103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:14.803662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:14.803662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/1e961cef8d2643c480253e713a38bdde/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:14.814509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, user_id=1e961cef8d2643c480253e713a38bdde, role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:14.816059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:14.851309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 67/133] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:14 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/1e961cef8d2643c480253e713a38bdde/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:14.852532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:14.899285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:15.077442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 67/134] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 822 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:15.078433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:15.086784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:15.182103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c805277-1e91-44f1-8ea4-ce589734e2e3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:15.284500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:15.286096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.287158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.287959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.291548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:15.294569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43247e-11ee-46f4-9d6e-ba0e24c7d725 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:15.298408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 68/135] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:14 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:15.300978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8c9f-470e-4716-b14d-3954f5f58de6 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.301789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8c9f-470e-4716-b14d-3954f5f58de6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.302974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc8c9f-470e-4716-b14d-3954f5f58de6 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.305799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 69/136] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:15.308001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc8c9f-470e-4716-b14d-3954f5f58de6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:15.349952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce29779-0233-493b-b8c5-6c1e97d773aa None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:15.421881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:15.423595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.424383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.425132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.427856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:15.431719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2101b-eee5-4979-85f8-5d1cea7ec01b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:15.435826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 68/137] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:15 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:15.438615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.439690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.440301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.473367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:15.475941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:15.494179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:15.531113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc8c9f-470e-4716-b14d-3954f5f58de6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:15.608609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:15.610428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:15.612051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.613369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.614047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.622975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 69/138] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:15.624755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:15.640593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:15.652189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:15.654178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:15.656153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:15.658058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 70/139] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:15.659754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:15.835211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add5649-1c96-4498-af7d-11d19ba9cb52 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:15.902225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:15.903810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:15.904555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:15.905337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] PATH_INFO: `/v3/projects/afb395674b73408f93f3a0fa436e2036` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:15.907772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] RBAC: Authorizing `identity:get_project(project_id=afb395674b73408f93f3a0fa436e2036)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:15.911036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:15.914585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 70/140] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:15 2022] GET /identity/v3/projects/afb395674b73408f93f3a0fa436e2036 => generated 312 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:15.921474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:15.957929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb11f-7bd3-4ac3-9d7f-842e8699f7fe None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.005452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.006091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.006376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.006647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.007512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=886d48441c5548498fa0b04224f8e56b) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:16.007829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.010096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.018016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 71/141] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:15 2022] GET /identity/v3/role_assignments?user.id=886d48441c5548498fa0b04224f8e56b&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:16.019126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.034100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:16.071572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:16.153677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4212da-60db-4f99-9a49-084eced5913d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.208346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d360-5312-4bf7-80a6-82cd2ff4a15b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.266800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.267927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.268424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.268906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.269832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:16.270338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.272846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.300115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 71/142] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:16.300856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.342241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.343206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.343642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.344108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] PATH_INFO: `/v3/projects/afb395674b73408f93f3a0fa436e2036/users/1e961cef8d2643c480253e713a38bdde/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.347936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=afb395674b73408f93f3a0fa436e2036, user_id=1e961cef8d2643c480253e713a38bdde, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.352641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.388931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 72/143] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:16 2022] PUT /identity/v3/projects/afb395674b73408f93f3a0fa436e2036/users/1e961cef8d2643c480253e713a38bdde/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:16.390454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.467101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ab220-448e-4688-baa4-99e29bb6843c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.519353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:16.527033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.527852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.528155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.531849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.546907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:16.562713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:16.565031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:16.566773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 72/144] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:16.571341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.587586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:16.698791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5140c3-576c-4a95-aaa1-6bc20debe2c6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.756848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c87892-2da4-4bbc-8df1-06fc1260642a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:16.813022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.814206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.814688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.815172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.820151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.820888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:16.821817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:16.822241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:16.822654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:16.823040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.823888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:16.824362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:16.825440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 73/145] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:16.828614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:16.842577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 73/146] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:16.850734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.851388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:16.963867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:17.059388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2304b-59da-4474-92d5-39ef5259114c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:17.129707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b3935-c652-4755-b6d7-5d8bf71e4624 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:17.162655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:17.163354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:17.163636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:17.163908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] PATH_INFO: `/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:17.165253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f809fdc6b04f0ca4d4c2bc46e0a712)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:17.166867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:17.168540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 74/147] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/users/f9f809fdc6b04f0ca4d4c2bc46e0a712 => generated 247 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:17.177072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:17.241251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:17.242086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:17.242245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:17.242616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:17.260074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:17.406010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:17.407839 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8483422-7ad3-471b-835a-a669a1f0e488 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:17.451253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf360c7-531e-4bca-898f-0554b898cca4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:17.519806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:17.572301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:17.573329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:17.573775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:17.574201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:17.763665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 74/148] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 913 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:17.765771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f5e2-e4e6-4498-b5ac-d0213c791216 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:17.766193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f5e2-e4e6-4498-b5ac-d0213c791216 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:17.766440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:17.766609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f5e2-e4e6-4498-b5ac-d0213c791216 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:17.769370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 75/149] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:17.771705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f5e2-e4e6-4498-b5ac-d0213c791216 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:17.775501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278018-b054-42c5-a6ba-66a09ad0c712 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:17.864934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:18.066295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 75/150] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 889 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:18.069006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:18.069509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:18.069985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:18.077022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f5e2-e4e6-4498-b5ac-d0213c791216 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:18.132068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:18.134159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:18.187522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:18.201541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:18.202652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:18.203137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:18.205288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:18.208988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:18.211751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:18.215101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 76/151] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:17 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:18.224561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:18.343824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 76/152] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:18.344833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:18.490574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f78fd2-063b-453f-9702-0396f54ac6b2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:18.493403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:18.561605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:18.562337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:18.562617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:18.562881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:18.568137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:18.569785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:18.572060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 77/153] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:18 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:18.580669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:18.678641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163dc88-f9eb-4a04-9ed2-f1a960cb2dfc None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:18.766032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985002a3-27c6-4e7d-b162-be63e4dcb660 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:18.826000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:18.827028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:18.827543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:18.828023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:18.829404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:18.829890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:18.832499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:18.832780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:18.833552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:18.834039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:18.834533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:18.836369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:18.836910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:18.844027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d6bf7-b987-422f-8cb4-e4128a6b3021 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:18.855858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 77/154] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:18 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=cc467837f6074b9391a11b0e57406d12 => generated 608 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:18.864813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 78/155] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:18 2022] GET /identity/v3/role_assignments?user.id=f9f809fdc6b04f0ca4d4c2bc46e0a712&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:19.067688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:19.102282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:19.202770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9614-29bd-4975-9c4f-052d2cf2c62a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:19.203246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9614-29bd-4975-9c4f-052d2cf2c62a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:19.203734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9614-29bd-4975-9c4f-052d2cf2c62a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:19.211295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 79/156] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:19.225727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:19.226198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:19.226674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:19.283505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:19.285526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:19.294474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9517fc-3b95-4e3f-a53b-e369d0f586c0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:19.370330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:19.379776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:19.380777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:19.381166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:19.381543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:19.383601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:19.385413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:19.387376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 78/157] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:19.393365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:19.466320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 80/158] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:19.507898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:19.534464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374baef-7d4a-4ae6-997f-cb454441f2f1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:19.577267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:19.577916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:19.578200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:19.578474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:19.579179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:19.580739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b2c22-dd4b-468d-8bca-192a0ca2590b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:19.613877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 79/159] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:19 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:38:19.875944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:20.028353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44214123-fe38-4ea2-a52e-d382cc802b3b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:20.076321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd98ef-c10f-4087-8fe8-b9b6377bef84 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.077047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd98ef-c10f-4087-8fe8-b9b6377bef84 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.077047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd98ef-c10f-4087-8fe8-b9b6377bef84 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.078502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 80/160] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:20.084248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.084512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.084851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.097807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:20.099355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.100050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.100754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.111631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:20.114080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:20.121065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:20.129525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:20.130411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:20.131301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 81/161] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:20.137347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:20.177141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:20.221748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 81/162] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:20.281644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb732f5-4cb2-44d8-ac47-ee2c5a459ca0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:20.347058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:20.348253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.348895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.349369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.350302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:20.350814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:20.352790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3bb1e-c325-4607-b41f-8588adb265fc None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:20.364825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 82/163] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:20.370627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:20.457873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:20.542256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f25ae-e988-480c-b666-4a96989d6892 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:20.567856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c85e1-f49f-4a31-84aa-becebfb6d886 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.568242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c85e1-f49f-4a31-84aa-becebfb6d886 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.568589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c85e1-f49f-4a31-84aa-becebfb6d886 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.570509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 83/164] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:20.581528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.581971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.582523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.599976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:20.600429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.600667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.600896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.602111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:20.603650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:20.605509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 82/165] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:20 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:20.612759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:20.626647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:20.629843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:20.735156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 84/166] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:20.778350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:20.814485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1fd4-039e-4f2a-bf64-55381d348f60 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:20.824289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:20.889199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:20.890546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:20.891865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:20.892328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:20.898259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:20.944010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c23b4b-1feb-43e3-8b7a-6d52d71acd57 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:21.024104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:21.025245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:21.025730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:21.026271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:21.037591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:21.041289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:21.044993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:21.050716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:21.053563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:21.054512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:21.063206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 85/167] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:21.072053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.247992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:21.266632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 83/168] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:21.276401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.304276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd928b-bfd9-467d-a25c-c126b3f94725 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:21.320354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:21.393103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:21.393950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:21.394324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:21.394691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:21.395517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:21.395923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:21.397523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:21.399606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68971e-46e5-4bd4-ae79-c2dced4dd85b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:21.414827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 86/169] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:21.423493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.440293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:21.441213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:21.441709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:21.442128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:21.443728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:21.445374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:21.447364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 84/170] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:21.454096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.573692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c0fad-2233-4329-8608-33719f343902 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:21.575801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:21.611064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82d438-7785-48f1-82de-8211febe8a67 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:21.624733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:21.625747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:21.626224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:21.626700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:21.627754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:21.628377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:21.630093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:21.645426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 85/171] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:21.646727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.702405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:21.703547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:21.704131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:21.704594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] PATH_INFO: `/v3/users/1e961cef8d2643c480253e713a38bdde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:21.706118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] RBAC: Authorizing `identity:get_user(user_id=1e961cef8d2643c480253e713a38bdde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:21.707953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:21.710459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 87/172] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/users/1e961cef8d2643c480253e713a38bdde => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:21.736718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:21.759795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:21.988997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ff870-ad7c-4b0a-a7e4-29a327c56e5d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:22.029069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5215-94ce-462c-9d6a-fcffd671e0fb None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:22.077706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:22.078725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:22.079187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:22.079742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:22.169924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:22.171047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:22.171586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:22.172354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:22.220106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:22.237839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:22.238799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:22.239660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:22.240361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 86/173] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 594 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:22.250278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:22.264353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:22.267078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:22.307808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:22.339476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:22.503866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8557-ce53-4280-aaab-fbe0880f6b89 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:22.569808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:22.570882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:22.571422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:22.571935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:22.573284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:22.573831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:22.576826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:22.594622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 87/174] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:22.615519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:22.659669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 88/175] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 924 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:22.672179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:22.882197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcde49-0383-4d9c-8614-8c17c815f210 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:22.903690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:22.931106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:22.950896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:22.952055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:22.952536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:22.953022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] PATH_INFO: `/v3/projects/afb395674b73408f93f3a0fa436e2036` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:22.955333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] RBAC: Authorizing `identity:get_project(project_id=afb395674b73408f93f3a0fa436e2036)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:22.958104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:22.961563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 89/176] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:22 2022] GET /identity/v3/projects/afb395674b73408f93f3a0fa436e2036 => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:22.971482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:22.976481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0fa30-9bd8-43f3-a858-dacdedeca5aa None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.063625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.064680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.065224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.065702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] PATH_INFO: `/v3/users/886d48441c5548498fa0b04224f8e56b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.067317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] RBAC: Authorizing `identity:get_user(user_id=886d48441c5548498fa0b04224f8e56b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.071632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.071632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 88/177] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:22 2022] GET /identity/v3/users/886d48441c5548498fa0b04224f8e56b => generated 289 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:23.093104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:23.171350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f5361-2e25-4429-8a61-b0876442a331 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.275045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.276196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.276638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.277061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.278308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=afb395674b73408f93f3a0fa436e2036, user.id=1e961cef8d2643c480253e713a38bdde) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:23.278813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.281634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.311854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 90/178] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:22 2022] GET /identity/v3/role_assignments?user.id=1e961cef8d2643c480253e713a38bdde&scope.project.id=afb395674b73408f93f3a0fa436e2036&role.id=cc467837f6074b9391a11b0e57406d12 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:23.340747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e94b31-aba2-4432-9cd9-9c17203488f4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.416824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.417966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.418449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.418909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.422854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.425564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d5c4-8e54-4b6e-b154-f1b94dfc4bae None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.429294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 89/179] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:23 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:23.438907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:23.552509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:23.588381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e547d-67b4-435e-b493-ad41e1bcdf6f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.588838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e547d-67b4-435e-b493-ad41e1bcdf6f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.589212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e547d-67b4-435e-b493-ad41e1bcdf6f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.590791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 90/180] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:23.597828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd69a-c863-47aa-bbf6-3a26e0fe739b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.598268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd69a-c863-47aa-bbf6-3a26e0fe739b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.598648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd69a-c863-47aa-bbf6-3a26e0fe739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.628497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-215cd69a-c863-47aa-bbf6-3a26e0fe739b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:23.630035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-215cd69a-c863-47aa-bbf6-3a26e0fe739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:23.672100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:23.707978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416535e2-b367-43d7-bed0-3faaa9cbc539 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:23.781997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:23.783343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.783831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.784300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/886d48441c5548498fa0b04224f8e56b/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.788421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, user_id=886d48441c5548498fa0b04224f8e56b, role_id=b5f5ea897d5c4c0c81eb4ac24796b7c9)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:23.791408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208669e-2b4f-4efc-aa75-572f768a79b3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:23.794879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 91/181] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:23.822484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 91/182] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:23 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/users/886d48441c5548498fa0b04224f8e56b/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:23.900990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635abd-31a4-4214-9c08-a4e89b4814ea None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.901507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635abd-31a4-4214-9c08-a4e89b4814ea None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.901948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635abd-31a4-4214-9c08-a4e89b4814ea None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.903626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 92/183] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:23.915906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a3a3-d0f8-4487-9f8c-05ab886d1324 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:23.916597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a3a3-d0f8-4487-9f8c-05ab886d1324 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:23.917137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66a3a3-d0f8-4487-9f8c-05ab886d1324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:23.960306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0f66a3a3-d0f8-4487-9f8c-05ab886d1324 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:23.961577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f66a3a3-d0f8-4487-9f8c-05ab886d1324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:24.049285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 92/184] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:24.272772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ae26-dd2f-444c-ab69-9f6bb5296fed None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.273010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ae26-dd2f-444c-ab69-9f6bb5296fed None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.273173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2ae26-dd2f-444c-ab69-9f6bb5296fed None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.274905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 93/185] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:24.281068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.281565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.281639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.301330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:24.302952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:24.330795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:24.342275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2ae26-dd2f-444c-ab69-9f6bb5296fed None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.387443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 93/186] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:24.387443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.490394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2ae26-dd2f-444c-ab69-9f6bb5296fed None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:24.535933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:24.536841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.537261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.537734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.538064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461628d-1b63-4c9d-b88f-295e7f8ad8bb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:24.539268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:24.541139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:24.543107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 94/187] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:24.548998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.590745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:24.591581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:24.592117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.592534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.592938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.607490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:24.612791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:24.613846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:24.614810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:24.617701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 94/188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:24.623760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.654509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:24.726283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79250c-d2da-45ba-8fb7-8030269415bb None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:24.768250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:24.768881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.769161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.769426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.770107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:24.771617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:24.775283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45428e29-4467-460d-b889-5ba31c501611 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:24.801198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 95/189] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:24 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:38:24.803183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.827436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:24.828341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:24.828759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:24.829198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:24.830276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:24.830922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:24.834423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:24.860171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 95/190] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:24.866208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:24.926303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:24.967188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f30e1e-060a-4c8b-aa26-9e7dbd3aabb0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:24.987748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:25.015570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c71b0-d5a5-4959-a018-bb700186b723 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.027925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.029288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.029842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.030337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.045233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:25.051064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:25.052166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:25.053051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:25.056580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 96/191] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:25.065821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:25.070520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.071773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.071773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.071995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.094302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:25.095847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:25.097997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 96/192] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:24 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:25.104373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:25.236938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10c882-a575-4fc7-a52e-c217a56c27ae None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.264188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf75da-2f62-4799-be76-359a5e870205 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.287530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.288863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.289370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.289714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.290588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:25.291023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:25.292920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:25.301770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:25.309035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 97/193] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:25.317541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:25.334994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.336868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.337303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.337743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.359647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:25.519468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:25.521053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:25.527172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f138-1998-4232-a508-f01e57b9a127 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.592964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.594112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.594589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.595057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.597168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:25.599910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:25.602876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 98/194] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:25.611601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:25.706077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 97/195] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:25.726788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:25.769529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:25.787667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:25.841044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44809-bc97-4dce-8f35-3c888e340f03 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.918540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423785c-0ee0-4427-89f9-e5060b26d2f9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:25.962072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.962603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:25.963029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.963404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.963669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.963817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:25.964160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:25.964633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:25.964923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:25.966543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:25.968758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 98/196] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:25.980350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:26.155701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:26.158158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:26.256239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f737-b7d1-4c12-9823-ae9b270aff84 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:26.295595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:26.361567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:26.362231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:26.362478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:26.362706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:26.364396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=59c0ad455c7f4e0fae5a06fd1fbabf41) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:26.365337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:26.369801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:26.372622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:26.381585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 99/197] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/role_assignments?user.id=59c0ad455c7f4e0fae5a06fd1fbabf41&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:26.465674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 99/198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 855 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:26.472445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:26.634193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb942-9e63-4986-b3bf-b0e776ff1beb None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:26.700387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:26.701381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:26.701810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:26.702227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:26.708625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:26.711350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe1571-f758-46e5-aed7-5218df6e9f75 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:26.714406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 100/199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:26 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:26.723880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:26.836570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7da482-3fff-4706-95d4-f26cf88941a4 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:26.837064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7da482-3fff-4706-95d4-f26cf88941a4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:26.837613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7da482-3fff-4706-95d4-f26cf88941a4 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:26.840505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 101/200] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:26.856730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e62be-bc89-40f2-b8ed-f984ec541aa4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:26.857166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e62be-bc89-40f2-b8ed-f984ec541aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:26.857590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e62be-bc89-40f2-b8ed-f984ec541aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:26.898531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed385-c4b4-46bf-a940-5a8f0a51f241 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:26.923242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e95e62be-bc89-40f2-b8ed-f984ec541aa4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:26.925332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e95e62be-bc89-40f2-b8ed-f984ec541aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:26.973942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:26.974908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:26.975334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:26.975778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:26.977042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=59c0ad455c7f4e0fae5a06fd1fbabf41) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:26.977479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:26.980178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:27.006947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 100/201] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:26 2022] GET /identity/v3/role_assignments?user.id=59c0ad455c7f4e0fae5a06fd1fbabf41&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:27.018191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:27.105881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 102/202] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:27.705009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:27.744045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:27.836944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf9468-1633-4ba2-89ef-b4a0d9517db6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:27.844683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54efa79-e027-4324-b957-e3846f0cd346 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:27.846061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54efa79-e027-4324-b957-e3846f0cd346 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:27.846749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54efa79-e027-4324-b957-e3846f0cd346 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:27.849398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 103/203] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:27.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf694bd-388c-4580-8501-49367ef16279 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:27.866754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf694bd-388c-4580-8501-49367ef16279 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:27.867510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf694bd-388c-4580-8501-49367ef16279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:27.913312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-abf694bd-388c-4580-8501-49367ef16279 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:27.917401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-abf694bd-388c-4580-8501-49367ef16279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:27.918849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:27.920270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:27.921015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:27.921745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:27.961039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:27.973940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:27.975685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:27.977329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:27.987782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 101/204] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:27.996791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:28.201261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 104/205] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:28.260000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:28.268535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eff0b-530d-4c49-980f-c7748b719e8f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:28.330099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:28.331131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:28.331652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:28.332129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:28.333331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:28.333869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:28.336456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:28.354140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 102/206] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:28.360153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf694bd-388c-4580-8501-49367ef16279 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:28.561129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:28.566242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf694bd-388c-4580-8501-49367ef16279 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:28.589204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:28.675005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:28.676100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:28.676576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:28.677049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] PATH_INFO: `/v3/users/886d48441c5548498fa0b04224f8e56b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:28.680280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] RBAC: Authorizing `identity:get_user(user_id=886d48441c5548498fa0b04224f8e56b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:28.683351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:28.686547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 105/207] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:28 2022] GET /identity/v3/users/886d48441c5548498fa0b04224f8e56b => generated 289 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:28.691883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:28.838876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad3235-0aaa-4b19-a0c7-af39cb5e233a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:28.854751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22707e0f-eda2-4af8-a2af-34cf33a92b64 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:28.904069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:28.905871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:28.907041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:28.908197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:28.924807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:28.925847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:28.926317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:28.926792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:28.948544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:28.955885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:28.956834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:28.957675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:28.966231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 103/208] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:28.975181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:29.011926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.013334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.083987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:29.085561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99165f5-743b-430e-af28-f21d562801d6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:29.206361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a537595-beb1-4c35-8d35-2f6d7eb71ecc None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:29.257254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 106/209] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:29.260579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab08aa-1f72-4f7f-8c07-b42f9a0f5ed2 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.261037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab08aa-1f72-4f7f-8c07-b42f9a0f5ed2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.261495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab08aa-1f72-4f7f-8c07-b42f9a0f5ed2 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.265374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 107/210] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:29.266034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab08aa-1f72-4f7f-8c07-b42f9a0f5ed2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:29.288564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:29.289211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.289488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.289758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.290468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:29.290772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:29.292547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388fc99-01c2-462c-8ed2-cebad3ee3ca9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:29.302856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 104/211] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:29.304364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7947-a429-4002-83dc-d0a7cea28047 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.304648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7947-a429-4002-83dc-d0a7cea28047 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.304904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7947-a429-4002-83dc-d0a7cea28047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.326525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3d6d7947-a429-4002-83dc-d0a7cea28047 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:29.328235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3d6d7947-a429-4002-83dc-d0a7cea28047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:29.393956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 105/212] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb277f7-3ed3-4afd-bd43-f97422346190 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb277f7-3ed3-4afd-bd43-f97422346190 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb277f7-3ed3-4afd-bd43-f97422346190 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 106/213] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:29.418460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb277f7-3ed3-4afd-bd43-f97422346190 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:29.434290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.479024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab08aa-1f72-4f7f-8c07-b42f9a0f5ed2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:29.547819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:29.548754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.549190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.549587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.551097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:29.552890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438ca31-1b53-4ce9-ac5e-add4b76c1bcd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:29.556056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 108/214] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:29 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:29.558403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.559245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.559764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.582243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:29.583914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:29.658291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb277f7-3ed3-4afd-bd43-f97422346190 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:29.661346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 109/215] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:29.668646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:29.737949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:29.738932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.740973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.741504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] PATH_INFO: `/v3/users/ca037f9060c34af4b2a509d657ffbd8d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.775036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] RBAC: Authorizing `identity:get_user(user_id=ca037f9060c34af4b2a509d657ffbd8d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:29.778610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:29.787248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 107/216] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:29 2022] GET /identity/v3/users/ca037f9060c34af4b2a509d657ffbd8d => generated 297 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:29.792622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:29.834182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.926350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b627e-919a-4725-bedb-492190feffe0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:29.953374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:29.992045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:29.992945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:29.993390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:29.993835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:29.995457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c, user.id=886d48441c5548498fa0b04224f8e56b) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:29.996404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:29.998923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.011770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 110/217] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:29 2022] GET /identity/v3/role_assignments?user.id=886d48441c5548498fa0b04224f8e56b&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:30.013604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.021357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48e511-349d-4971-a631-d1af5bb20648 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.094747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.095470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.095857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.096167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.214790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0ec7-a520-4722-8111-7681d45d52bf None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.221169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.224515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5d614-dcb5-457b-a098-7a802385329f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.257609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:30.267084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.268222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.268808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.269157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.270856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.272446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.274476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 111/218] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:30.277433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.310037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:30.352875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 108/219] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:30.354283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425353ba-c057-4f2d-bd37-2c2f3486b075 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.354553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425353ba-c057-4f2d-bd37-2c2f3486b075 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.354813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425353ba-c057-4f2d-bd37-2c2f3486b075 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.356384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 109/220] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:30.356948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425353ba-c057-4f2d-bd37-2c2f3486b075 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.408034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb546-4725-4621-a948-2e7c37178290 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.453470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.454493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.454932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.455405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.468290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:30.473873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:30.475371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 112/221] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:30.476154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.484946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425353ba-c057-4f2d-bd37-2c2f3486b075 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.526883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.527448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.527729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.528016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.528689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.530173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab3ee2d-12be-4cf8-8323-9d1fe3b8166c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.547924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:30.554690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 110/222] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:30 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:30.556669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.556944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.557219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.582826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:30.584454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:30.604965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:30.639009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122957d-51b2-4ed4-bb62-0a6184349f4e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.701159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 111/223] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:30.701159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.709912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.711122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.711831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.712263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.714787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.717874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.721540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 113/224] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:30.731802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:30.877789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795f828-c2ff-4194-badb-633eccd1bd13 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.917133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae030e8-8b1b-4c35-941f-1e58e3cc5c5f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:30.947095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.948595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.949293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.949985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.951429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:30.952239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.954132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:30.955750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.964705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:30.965434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:30.965762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:30.966090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:30.966911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=ca037f9060c34af4b2a509d657ffbd8d) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:30.967313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:30.968959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:30.974769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 112/225] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:30.976549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 114/226] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/role_assignments?user.id=ca037f9060c34af4b2a509d657ffbd8d&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=cc467837f6074b9391a11b0e57406d12 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:30.983579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.005843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:31.155990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.187314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5a88b-aee1-4965-a162-a82bc6dbc1ea None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:31.264801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:31.266074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:31.266379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:31.266871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:31.269190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:31.273146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:31.277325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 113/227] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:30 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:31.284617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.443739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730343-e8dc-4e55-9d7f-a10c062c5a59 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:31.502671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:31.536946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:31.537631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:31.537905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:31.538175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:31.540851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465800d5-1990-43e1-a3ca-6f4b8d62a0f0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:31.576827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:31.594215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:31.595977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:31.597527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:31.600982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 115/228] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:31.610866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.650985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:31.651665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:31.651939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:31.652207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:31.654823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:31.656329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:31.658039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 114/229] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:31 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:31.667661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.758047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:31.836190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527154f-f787-414f-9f05-72579c48f953 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:31.868454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a02b1e-2a0f-4283-ba29-cf7a815e2aa5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:31.908643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:31.909347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:31.909728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:31.909997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/59c0ad455c7f4e0fae5a06fd1fbabf41/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:31.917183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, user_id=59c0ad455c7f4e0fae5a06fd1fbabf41, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:31.918898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:31.933151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:31.934206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:31.934565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:31.935633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:31.939657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:31.940030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:31.944722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 115/230] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:31 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/59c0ad455c7f4e0fae5a06fd1fbabf41/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:31.945088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:31.957572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 116/231] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:31.963900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:31.979865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:32.172800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267e5b6-7600-4f5f-9c50-2e73324ce2be None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:32.246957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:32.248018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:32.248489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:32.248957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:32.250904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:32.253384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:32.258769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 116/232] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:31 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:32.268486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:32.485181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a45b3-7ffb-46e7-a295-d92dd9a2db0e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:32.514976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:32.554760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899e15d-935d-474d-b222-81843943b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:32.556792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899e15d-935d-474d-b222-81843943b93d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:32.557561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899e15d-935d-474d-b222-81843943b93d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:32.558301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899e15d-935d-474d-b222-81843943b93d None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:32.560986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899e15d-935d-474d-b222-81843943b93d None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:32.564467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899e15d-935d-474d-b222-81843943b93d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:32.568780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 117/233] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:32 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:32.574891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:32.749247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:32.801200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93aaf26-664a-444b-afe1-1f8e93823198 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:32.886746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:32.887888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:32.888377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:32.888864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/59c0ad455c7f4e0fae5a06fd1fbabf41/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:32.896626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, user_id=59c0ad455c7f4e0fae5a06fd1fbabf41, role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:32.899286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d1162-f30b-4f0d-9a0a-997fc3886e3d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:32.933591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 117/234] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:32 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/59c0ad455c7f4e0fae5a06fd1fbabf41/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:38:34.419523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26778-d70d-4f6a-955c-080b91afc4a3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.420622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26778-d70d-4f6a-955c-080b91afc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.421105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d26778-d70d-4f6a-955c-080b91afc4a3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.423243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 118/235] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:34.428868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02eb2-0dc5-4769-8204-a1871743f9fd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.429826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02eb2-0dc5-4769-8204-a1871743f9fd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.430665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec02eb2-0dc5-4769-8204-a1871743f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.457396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1ec02eb2-0dc5-4769-8204-a1871743f9fd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:34.459051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1ec02eb2-0dc5-4769-8204-a1871743f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:34.569226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 118/236] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:34.584743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63abd-727e-42c0-b842-f2cabf975cf8 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.585659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63abd-727e-42c0-b842-f2cabf975cf8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.586425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b63abd-727e-42c0-b842-f2cabf975cf8 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.589844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 119/237] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:34.599594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.600557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.601301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.633078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:34.635865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:34.658209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69613-dfe5-4813-81c5-c3a8786989a3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.659246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69613-dfe5-4813-81c5-c3a8786989a3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.659984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69613-dfe5-4813-81c5-c3a8786989a3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.662978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 120/238] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:34.670387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:34.671327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:34.672105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:34.723536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:34.727333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:34.827643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:34.843193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 119/239] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:34.900258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 121/240] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:34.962967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.168510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ed63-10d3-4520-a0c7-b19a97ab6ee1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:35.195778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.261095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:35.261854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.262140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.262414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.288012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:35.294664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:35.296304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:35.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:35.301386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 120/241] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:35.312242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.446518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:35.470763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97347b82-54a1-40e0-a2cb-2deb9aacfba6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:35.500700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:35.541874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7677cf-d312-43ff-bfa4-8f3345897929 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:35.550644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:35.552119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.552764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.553402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.556049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:35.559239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:35.562884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 122/242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:35.564878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.608239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:35.609063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.609424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.609914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.611257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:35.611729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:35.614384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:35.637918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 121/243] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:35.637918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.748414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8842-d856-4e47-91c8-9cc3a811307f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:35.793779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:35.794421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.794762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.795096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.809169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7ab35606-377d-4bb1-8f69-e5cfffed343f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:35.814581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:35.816329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 123/244] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:35.817832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c2ad8-c7f0-42d6-9c5a-d4bb97070bb2 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.829222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c2ad8-c7f0-42d6-9c5a-d4bb97070bb2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.829222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c2ad8-c7f0-42d6-9c5a-d4bb97070bb2 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.829222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 124/245] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:35.829222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c2ad8-c7f0-42d6-9c5a-d4bb97070bb2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.830601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d84d1-af45-4168-83d9-a78ffde42130 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:35.851041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:35.874776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:35.875467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:35.875753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:35.876034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:35.876893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:35.878399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:35.883570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:35.905443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 122/246] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:35 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:35.908244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:35.993673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c2ad8-c7f0-42d6-9c5a-d4bb97070bb2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.046716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.050585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.051025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.051513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] PATH_INFO: `/v3/users/ca037f9060c34af4b2a509d657ffbd8d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.053811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] RBAC: Authorizing `identity:get_user(user_id=ca037f9060c34af4b2a509d657ffbd8d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.061431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb9aed-df71-4cf2-98d1-c99a2ff44b83 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.073416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 125/247] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:35 2022] GET /identity/v3/users/ca037f9060c34af4b2a509d657ffbd8d => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:36.075847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.076364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.076884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.099582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f5ac5-3c0e-4b19-abbf-9cc1e27fc337 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.109211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:36.110993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:36.147866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.149216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.149550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.150003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.151332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:36.151635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.153540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.165311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 123/248] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:36.165992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:36.217519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 126/249] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:36.219440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:36.251624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:36.261152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:36.339720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ced00-568d-454e-a8d1-cb8fbb2ffeb2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.365042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607a572-87e0-4ecc-a243-076fd3197b5c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.397234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.398027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.398299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.398594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.400335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.402185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.404956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 124/250] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:36 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:36.412392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.413154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.413436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.413707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] PATH_INFO: `/v3/users/fc2983baa0734400abf3ec790ca64aa0` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.414659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:36.436469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] RBAC: Authorizing `identity:get_user(user_id=fc2983baa0734400abf3ec790ca64aa0)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.436469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.436754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 127/251] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:36 2022] GET /identity/v3/users/fc2983baa0734400abf3ec790ca64aa0 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:36.444836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:36.580187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:36.581309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4976985-93e8-4e90-a00c-f4fd537184e6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.585175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:36.590027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa044af9-b4b8-4d7f-831c-0cafefae26d9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:36.627941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.628725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.629123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.630340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/ca037f9060c34af4b2a509d657ffbd8d/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.632545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, user_id=ca037f9060c34af4b2a509d657ffbd8d, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.636513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc73f68-df92-44d9-8669-8bffe54e2362 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.636752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:36.636752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.636987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.637298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.662705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 125/252] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:36 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/ca037f9060c34af4b2a509d657ffbd8d/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:36.736580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:36.738151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:36.859778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:36.865629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a406-daed-411d-b4fe-f070d979e5c8 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.866076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a406-daed-411d-b4fe-f070d979e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.866549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a406-daed-411d-b4fe-f070d979e5c8 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.868765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 128/253] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:36.873361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 126/254] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:36.880603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:36.882439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:36.882929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:36.883745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:36.942424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:36.946224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:37.108957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57ef3-33cc-4ff3-a41f-4892fc43f524 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:37.121723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 127/255] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:37.125010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:37.152482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:37.153300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:37.153677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:37.154641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:37.155388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:37.157021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:37.159550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 129/256] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:36 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:37.165982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:37.312806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:37.443874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:37.460350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2141b5-c47e-4f09-a6fd-575e71daecd8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:37.483450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ef60-9eda-40e3-a673-a50d6a64fb0d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:37.544857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:37.552620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:37.553080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:37.553562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:37.577794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:37.579302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:37.580093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:37.580786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:37.583209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=fc2983baa0734400abf3ec790ca64aa0) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:37.584063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:37.587295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:37.594413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 130/257] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:37 2022] GET /identity/v3/role_assignments?user.id=fc2983baa0734400abf3ec790ca64aa0&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 259 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:37.595749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:37.612385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:37.622018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:37.622981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:37.623802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:37.632697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 128/258] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 509 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:37.642815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:37.733206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52392-8aab-49af-9423-65b81754ba8f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:37.765827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:37.777232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:37.778276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:37.778779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:37.779267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:37.807874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:37.823271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:37.824223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:37.824978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:37.828790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 131/259] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:37.836239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:37.926148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a3eff-5546-440e-ba5c-123f67a1c7bf None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:37.974275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:37.974934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:37.975211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:37.975499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:37.976199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:37.976497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:37.977926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:37.997178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 129/260] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:38.005620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:38.039912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:38.197836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4c06-f62b-4259-81eb-1606cb0f38f5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:38.205465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15013b-d206-4104-b683-5d8b4e7c1818 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:38.279846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:38.281144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:38.281632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:38.282551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:38.284129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:38.291253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:38.294190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 130/261] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:38 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:38.302656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:38.336097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:38.336755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:38.337041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:38.337369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:38.338160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:38.338497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:38.340073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:38.358651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 132/262] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:38.364691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:38.477453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:38.492513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:38.522558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ae3-1148-4366-9983-6a8eaf02057f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:38.575295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07f4c1-84b0-47d7-8094-0b61690d444e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:38.605953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:38.607548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:38.608087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:38.608548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:38.630829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:38.633596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:38.635165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:38.635867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] PATH_INFO: `/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:38.639727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] RBAC: Authorizing `identity:get_user(user_id=59c0ad455c7f4e0fae5a06fd1fbabf41)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:38.644812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:38.648143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 133/263] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:38 2022] GET /identity/v3/users/59c0ad455c7f4e0fae5a06fd1fbabf41 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:38.655329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:38.762808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:38.764372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:38.824152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee0a6d-656a-4570-b4f3-292a5ab42318 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:38.844205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:38.867383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:38.868044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:38.868328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:38.868600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:38.931132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:38.958822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 131/264] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:38.974576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:38.977442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:38.983829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:39.277105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788771bb-26e4-43ec-84f8-f223e7f50ff9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:39.328681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 134/265] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:39.334293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:39.357448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:39.358487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:39.358893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:39.359317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:39.363360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:39.365914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:39.369168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 132/266] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:38 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:39.373554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:39.376936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:39.541118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:39.613161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124d67-0a04-46ab-8eb6-7d39fbe6e364 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:39.660241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f747424-28ca-435d-92e6-8a6a1a628197 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:39.700004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:39.700649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:39.700926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:39.701200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:39.702438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:39.703975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:39.705931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 135/267] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:39 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:39.712314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:39.732347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:39.733499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:39.733980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:39.734468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:39.735826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=59c0ad455c7f4e0fae5a06fd1fbabf41) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:39.736328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:39.739014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cd0ac-bf93-4d63-b190-5e055b2ea50f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:39.760704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 133/268] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:39 2022] GET /identity/v3/role_assignments?user.id=59c0ad455c7f4e0fae5a06fd1fbabf41&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:39.873046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:39.898817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d31c9-779a-4f1a-a941-2d47ddd0b6c8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:39.957185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:39.957831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:39.958104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:39.958369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:39.959120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=59c0ad455c7f4e0fae5a06fd1fbabf41) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:39.959519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:39.961002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cafa93-79f7-403d-8204-5748809d974b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:39.971459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 136/269] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:39 2022] GET /identity/v3/role_assignments?user.id=59c0ad455c7f4e0fae5a06fd1fbabf41&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:40.098716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b873-ee2b-4ce8-a230-26bd030e1bec None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.099297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b873-ee2b-4ce8-a230-26bd030e1bec None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.099297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b873-ee2b-4ce8-a230-26bd030e1bec None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.101037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 134/270] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:40.115544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a76-a1f1-4d09-8ee5-a72008633f2c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.116126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a76-a1f1-4d09-8ee5-a72008633f2c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.116405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c10a76-a1f1-4d09-8ee5-a72008633f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.144587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d0c10a76-a1f1-4d09-8ee5-a72008633f2c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:40.145743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d0c10a76-a1f1-4d09-8ee5-a72008633f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:40.261079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 137/271] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:40.267271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b183c8f-3600-4feb-a449-baf519553782 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.267619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b183c8f-3600-4feb-a449-baf519553782 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.267833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b183c8f-3600-4feb-a449-baf519553782 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.269390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 135/272] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:40.280105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.280762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.281231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.308905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:40.310659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:40.407373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 138/273] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:40.702154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b183c8f-3600-4feb-a449-baf519553782 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:40.835847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:40.848323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:40.855478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b183c8f-3600-4feb-a449-baf519553782 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:40.887825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:40.904413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:40.905548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:40.905906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:40.906249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:40.919777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:40.925289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:40.927896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:40.929146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 136/274] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:40.933990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:40.965153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8433044-2a21-4c90-86c1-dc707060917c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.005275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.005947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.006220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.006488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.007874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:41.009577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:41.016242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 139/275] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:41.017253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:41.049139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82b229-1a21-4718-abc7-733f648fed12 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.091683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.092660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.092999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.093416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.094513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:41.094992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:41.097457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1be31d-6a19-4cdc-8281-6ca5da700b98 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:41.113413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 137/276] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:41.115093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e12e3-c01d-451f-84a9-77a7d10edbf3 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.115466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e12e3-c01d-451f-84a9-77a7d10edbf3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.115800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e12e3-c01d-451f-84a9-77a7d10edbf3 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.117283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 138/277] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:41.121625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf589b-c0a9-4ea0-b3fa-0e308d0b14e6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.122094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf589b-c0a9-4ea0-b3fa-0e308d0b14e6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.122563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf589b-c0a9-4ea0-b3fa-0e308d0b14e6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.124388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 139/278] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:41.125574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf589b-c0a9-4ea0-b3fa-0e308d0b14e6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:41.141556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:41.169649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec6f08-f6d1-4422-9a25-33d001e1d025 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.178032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:41.211284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.211995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.212424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.212854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.213633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:41.215357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beac5c4-dd05-4d18-a35c-682ec66e827e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:41.241192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 140/279] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:38:41 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:41.242747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.243202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.243582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.265923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf589b-c0a9-4ea0-b3fa-0e308d0b14e6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.270254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:41.272130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:41.316363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.317446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.317874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.318281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] PATH_INFO: `/v3/users/27f6c2bbb05c432a95f0eee7143e370b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.339947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c2bbb05c432a95f0eee7143e370b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:41.341544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c670-8737-4d2d-84b6-748b5dfdaa09 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:41.343616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 140/280] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:41 2022] GET /identity/v3/users/27f6c2bbb05c432a95f0eee7143e370b => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:41.349766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.350064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.350399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.373172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:41.375099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:41.390315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 141/281] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:41.391773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:41.517272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 141/282] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:41.528822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:41.609944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fbc4-d31b-47cf-8769-7c04fe8962f4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.676929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.677913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.678335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.678741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.680409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, user.id=27f6c2bbb05c432a95f0eee7143e370b, scope.system=all) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:41.680904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:41.683704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:41.687516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:41.693545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 142/283] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Sep 12 23:38:41 2022] GET /identity/v3/role_assignments?user.id=27f6c2bbb05c432a95f0eee7143e370b&scope.system=all&role.id=cc467837f6074b9391a11b0e57406d12 => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:41.723836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:41.803448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:41.841847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424d290-ecb8-493c-8030-054d909cef19 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:41.901213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:41.902265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:41.903574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:41.903986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:41.918129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:41.923527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:41.925176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:41.926641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 142/284] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:41.936349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.029078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eab34-6ea8-4767-9317-e08869f61493 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.071101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:42.095114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:42.104680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dd639-1ff0-4bbf-a562-d14fd10c4766 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.118831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.119527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.119798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.120069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.144480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:42.150021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:42.150945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:42.151764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:42.160255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 143/285] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:42.161246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.183563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.184239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.184467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.184744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.185423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:42.185723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:42.187169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:42.197199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 143/286] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:42.213429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.316364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3fa8d-676b-462c-92af-f0267a398c5f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.358649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.359683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.360185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.360654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.362347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:42.362916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:42.364606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:42.376060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 144/287] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:42.382353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.451666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:42.465520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef8a9-6e56-4b12-baa4-c51e75ee97de None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.509829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32192b10-cf36-444b-b3f1-4c9b862fd437 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.550448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.551354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.551705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.552261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] PATH_INFO: `/v3/users/fc2983baa0734400abf3ec790ca64aa0` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.559252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] RBAC: Authorizing `identity:get_user(user_id=fc2983baa0734400abf3ec790ca64aa0)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:42.559252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:42.562059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 144/288] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:42 2022] GET /identity/v3/users/fc2983baa0734400abf3ec790ca64aa0 => generated 297 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:42.570500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.599488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.601348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.602477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.603560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] PATH_INFO: `/v3/users/ca037f9060c34af4b2a509d657ffbd8d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.605362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] RBAC: Authorizing `identity:get_user(user_id=ca037f9060c34af4b2a509d657ffbd8d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:42.608044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:42.610365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:42 2022] GET /identity/v3/users/ca037f9060c34af4b2a509d657ffbd8d => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:42.612307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:42.615778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.826342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55bc78-2acf-4c3e-a407-86024e2102a7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.881615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b4679-03ac-45b4-bf5d-b566de47671f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:42.903767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:42.908554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.909671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.910174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.910672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:42.914370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:42.917276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:42.927877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 145/290] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:42 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:42.931574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:42.938150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:42.938787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:42.939062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:42.939365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:43.097101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:43.100360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:43.120607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:43.153379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2de8-4205-41ad-ac97-1352bd58fec1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:43.229142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:43.231278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:43.232037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:43.234116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/fc2983baa0734400abf3ec790ca64aa0/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:43.237855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, user_id=fc2983baa0734400abf3ec790ca64aa0, role_id=b5f5ea897d5c4c0c81eb4ac24796b7c9)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:43.241179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:43.274895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 146/291] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:38:42 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/users/fc2983baa0734400abf3ec790ca64aa0/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:43.390554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 146/292] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:43.396917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:43.557191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1081-0823-4c3e-9fdb-c1594121b0d9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:43.597453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:43.598373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:43.598815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:43.599253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:43.600619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:43.602224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa8a4-8a0a-4400-9319-dddaf2b58e0d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:43.604176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 147/293] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:43 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:43.612395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:43.632090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:43.727457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ffbe8-ea63-4812-aeb6-1a27c8bf9cc3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:43.768595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:43.769272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:43.769553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:43.769817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:43.770639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=ca037f9060c34af4b2a509d657ffbd8d) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:43.770947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:43.772455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1a649-9393-443d-8f4e-ec10982c9b82 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:43.779944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 147/294] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:43 2022] GET /identity/v3/role_assignments?user.id=ca037f9060c34af4b2a509d657ffbd8d&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=cc467837f6074b9391a11b0e57406d12 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:44.317499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41f27-ccdf-4c0a-b1d6-a529b1cfca4c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:44.318070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41f27-ccdf-4c0a-b1d6-a529b1cfca4c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:44.318070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41f27-ccdf-4c0a-b1d6-a529b1cfca4c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:44.319696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 148/295] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:44.327182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:44.327707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:44.327707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:44.349633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:44.350870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:44.449811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 148/296] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:44.745756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41f27-ccdf-4c0a-b1d6-a529b1cfca4c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:44.767871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:44.944204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41f27-ccdf-4c0a-b1d6-a529b1cfca4c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:44.985418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:44.986454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:44.986953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:44.987499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:44.988883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:44.990580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cad519-68f0-4144-9352-624d581135fe None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:44.992596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 149/297] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:45.002170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.015197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e754-f698-4304-89ac-f2ee4a7bd762 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.015580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e754-f698-4304-89ac-f2ee4a7bd762 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.015772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910e754-f698-4304-89ac-f2ee4a7bd762 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.017468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 150/298] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:45.021209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:45.026080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37c09-01fb-4b15-940c-6ead05c618ea None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.026500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37c09-01fb-4b15-940c-6ead05c618ea None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.026754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c37c09-01fb-4b15-940c-6ead05c618ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.047476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-23c37c09-01fb-4b15-940c-6ead05c618ea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:45.048707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-23c37c09-01fb-4b15-940c-6ead05c618ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:45.127983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 151/299] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:45.129516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cb8e-d391-4181-80bc-31aa4f5b076d None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.129935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cb8e-d391-4181-80bc-31aa4f5b076d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.130360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cb8e-d391-4181-80bc-31aa4f5b076d None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.131929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 152/300] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:45.145045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.145336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.145661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.168184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:45.170360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:45.173772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:45.226481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cfd1c-99d0-4c4e-956e-d107235c3025 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:45.262153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 153/301] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:45.300507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:45.301648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.302139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.302619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] PATH_INFO: `/v3/groups` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.304049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:45.307333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:45.338304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 149/302] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Sep 12 23:38:44 2022] POST /identity/v3/groups => generated 229 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:38:45.467912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.553477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.600687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53fd01-7733-499a-9e71-d286ea472ef5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:45.638710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:45.651062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:45.651763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.652040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.652311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.667381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:45.673656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:45.674658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 154/303] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:45.675908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:45.681453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.692831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e7d25-877f-44e5-a177-265797ffffbb None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:45.744916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:45.746441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.746966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.747469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.765029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:45.773678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:45.774634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:45.775526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:45.779082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 150/304] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:45.786417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.814253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa3251-2b5a-4d54-85a8-819908c65e9e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:45.856111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:45.856761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:45.857079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:45.857308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:45.858028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:45.858332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:45.859914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:45.876039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 155/305] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:45.878747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:45.946426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a92bd-2c79-415f-9ee8-7dbb9e3d924d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:45.991048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:45.998439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:46.002510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:46.003777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.004240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.004685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.005564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:46.006048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:46.007820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:46.022706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 151/306] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:46.032168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:46.094063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51719005-5325-40ce-95bb-5a0a8768846c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:46.171425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68917578-bf8e-41bb-8f27-dcf6f2d7dd57 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:46.182800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:46.185149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.185858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.187087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] PATH_INFO: `/v3/users/27f6c2bbb05c432a95f0eee7143e370b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.190130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c2bbb05c432a95f0eee7143e370b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:46.193264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:46.198966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 156/307] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:45 2022] GET /identity/v3/users/27f6c2bbb05c432a95f0eee7143e370b => generated 293 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:46.205151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:46.219087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:46.219776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.220104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.220377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] PATH_INFO: `/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.237336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] RBAC: Authorizing `identity:get_user(user_id=a795c20ded0b4cd992c9474b9ddd3ce7)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:46.240166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae76cb-9d95-4eab-881e-d35621f63607 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:46.243884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 152/308] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:46 2022] GET /identity/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:46.246133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d72b9-1235-4b98-a5e0-5c00db5017f1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.246598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d72b9-1235-4b98-a5e0-5c00db5017f1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.247066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d72b9-1235-4b98-a5e0-5c00db5017f1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.249815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 153/309] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:46.250989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d72b9-1235-4b98-a5e0-5c00db5017f1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:46.327182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:46.389269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d72b9-1235-4b98-a5e0-5c00db5017f1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:46.416562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f330-2e46-4023-be8e-d9f6198cceb8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:46.432658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:46.433287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.433561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.433830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.434582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, user.id=a795c20ded0b4cd992c9474b9ddd3ce7, scope.system=all) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:46.434889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:46.436420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fde8d0-6065-40ac-9339-0cc3f81150bf None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:46.443000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 154/310] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Sep 12 23:38:46 2022] GET /identity/v3/role_assignments?user.id=a795c20ded0b4cd992c9474b9ddd3ce7&scope.system=all&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:46.452729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509b728-4b16-4088-ba25-955402c39668 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.453304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509b728-4b16-4088-ba25-955402c39668 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.453879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509b728-4b16-4088-ba25-955402c39668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.480965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:46.490706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3509b728-4b16-4088-ba25-955402c39668 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:46.492059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:46.493452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3509b728-4b16-4088-ba25-955402c39668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:46.494058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:46.494855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:46.495686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] PATH_INFO: `/v3/system/users/27f6c2bbb05c432a95f0eee7143e370b/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:46.498792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=27f6c2bbb05c432a95f0eee7143e370b, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:46.502165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:46.523568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 157/311] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Sep 12 23:38:46 2022] PUT /identity/v3/system/users/27f6c2bbb05c432a95f0eee7143e370b/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:46.649906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 155/312] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:46.920460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:47.115984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebcbe6-2a38-41f0-8171-15a7bcce0160 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:47.182950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:47.184539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.185266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.186005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:47.205887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:47.218273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:47.219977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:47.221324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:47.225217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 158/313] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:47.234212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509b728-4b16-4088-ba25-955402c39668 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:47.320396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:47.416014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509b728-4b16-4088-ba25-955402c39668 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:47.475481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:47.476418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.476890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.477350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:47.478559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:47.479082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:47.481484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e724e-aa09-44df-a633-41baad1f3f3c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:47.497881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 156/314] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:47.506859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:47.606499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:47.699517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43af27-0bfe-49b7-b054-5851f799b8b9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:47.754748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b12d6-9574-44b0-ae52-a70119b61847 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.755152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b12d6-9574-44b0-ae52-a70119b61847 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.755796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b12d6-9574-44b0-ae52-a70119b61847 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:47.758632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 157/315] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:47.761063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:47.761736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.762008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.762278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] PATH_INFO: `/v3/users/fc2983baa0734400abf3ec790ca64aa0` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:47.763918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] RBAC: Authorizing `identity:get_user(user_id=fc2983baa0734400abf3ec790ca64aa0)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:47.765390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:47.767273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 159/316] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:47 2022] GET /identity/v3/users/fc2983baa0734400abf3ec790ca64aa0 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:47.773031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8717-7354-44a1-951a-75277c891fca None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.773305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8717-7354-44a1-951a-75277c891fca None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.773579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec8717-7354-44a1-951a-75277c891fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:47.778149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:47.793308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8eec8717-7354-44a1-951a-75277c891fca None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:47.794517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8eec8717-7354-44a1-951a-75277c891fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:47.915067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 158/317] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:47.916385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237c8ee-eff6-4031-8a47-8080bca00bdc None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:47.956534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:47.979833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:47.981401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:47.982226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:47.982939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.116915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:48.120233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:48.189082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec8717-7354-44a1-951a-75277c891fca None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:48.229804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:48.308814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec8717-7354-44a1-951a-75277c891fca None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:48.332002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 160/318] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 12 23:38:48.335793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:48.364231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:48.380535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:48.384256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:48.384982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:48.385631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.388815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:48.393085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:48.398761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 159/319] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:38:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:48.404526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:48.495986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552283f-2339-4285-9c89-259d62db9115 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:48.534264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44561b-35c2-4b22-b891-283da5e768fd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:48.537929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:48.568601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:48.570205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:48.571178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:48.571961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.574225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:48.578154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:48.587502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 161/320] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:48 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:48.593677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:48.606557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:48.607867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:48.608468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:48.609139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] PATH_INFO: `/v3/groups` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.611336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:48.614983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ba690-01e9-4992-80a5-3d2b2853a5de None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:48.651678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 160/321] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Sep 12 23:38:48 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:38:48.784535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:48.806229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841ebd3-a514-47f0-a3a1-ba5e27deb43e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:48.875349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:48.876360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:48.876847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:48.877293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.878751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, scope.project.id=ade415429d4a434eb11812b5d5fe5722, user.id=fc2983baa0734400abf3ec790ca64aa0) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:48.879327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:48.882186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313690fd-b4ce-4a0a-9679-77d84b74d170 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:48.894233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 162/322] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:38:48 2022] GET /identity/v3/role_assignments?user.id=fc2983baa0734400abf3ec790ca64aa0&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:48.993638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6c5f-8029-438f-b23d-e5dccc4fced5 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:48.993927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6c5f-8029-438f-b23d-e5dccc4fced5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:48.994218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6c5f-8029-438f-b23d-e5dccc4fced5 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:48.995927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 161/323] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:49.002028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:49.002317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:49.002781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:49.025765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:49.027994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:49.151979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 163/324] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:49.491153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6c5f-8029-438f-b23d-e5dccc4fced5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:49.634463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6c5f-8029-438f-b23d-e5dccc4fced5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:49.686888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:49.688022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:49.688564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:49.689030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:49.707518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-93be4407-e89e-4d84-9c37-f77dd098816b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:49.713043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:49.713969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:49.714743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:49.717575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 162/325] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:49.723878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:49.777093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d119fa-94e6-49c7-847b-b258d6aaa898 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:49.777093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d119fa-94e6-49c7-847b-b258d6aaa898 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:49.777093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d119fa-94e6-49c7-847b-b258d6aaa898 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:49.777093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 163/326] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:38:49.782923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:49.783287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:49.783527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:49.823468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:49.827473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:49.856254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:49.871083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:49.898032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066f88a-7d51-41f3-a669-fd83d48361c6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:49.949705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:49.950484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:49.950774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:49.951063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:49.951828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:49.952143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:49.953659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:49.966683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 164/327] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:49.973014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:49.987343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 164/328] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:50.114579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32204a8-93ea-4467-85ff-05b862ddda4c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:50.153806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:50.154465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:50.154740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:50.155036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] PATH_INFO: `/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:50.156255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] RBAC: Authorizing `identity:get_user(user_id=a795c20ded0b4cd992c9474b9ddd3ce7)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:50.157750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:50.159863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 165/329] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:49 2022] GET /identity/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:50.166284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:50.322911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54571c63-4812-4344-9c70-4180b04a2cb1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:50.369239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:50.369938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:50.370216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:50.370493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] PATH_INFO: `/v3/system/users/a795c20ded0b4cd992c9474b9ddd3ce7/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:50.372236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a795c20ded0b4cd992c9474b9ddd3ce7, role_id=b5f5ea897d5c4c0c81eb4ac24796b7c9)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:50.373751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:50.387041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 165/330] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Sep 12 23:38:50 2022] PUT /identity/v3/system/users/a795c20ded0b4cd992c9474b9ddd3ce7/roles/b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:38:50.611893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:50.630698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:50.742491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14220c-184e-47b9-bbbc-63e321cd706e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:50.793386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:50.794059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:50.794398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:50.794671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:50.807707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:50.813280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:50.815410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:50.817196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 166/331] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:50.821037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:50.875534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:51.070395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b4b84-46c8-43fe-9ac4-1120a66d4e2f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:51.161814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:51.162002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:51.162398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:51.162865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:51.164085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:51.164598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:51.167241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:51.188428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 166/332] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:51.194394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:51.311838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:51.351556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f95170-9821-4d12-8ad1-b358d41d5504 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:51.394878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:51.395602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:51.395888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:51.396314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] PATH_INFO: `/v3/users/27f6c2bbb05c432a95f0eee7143e370b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:51.397637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] RBAC: Authorizing `identity:get_user(user_id=27f6c2bbb05c432a95f0eee7143e370b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:51.399159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1610-34f8-476d-ae66-2dc5b04872d8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:51.401307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 167/333] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:51 2022] GET /identity/v3/users/27f6c2bbb05c432a95f0eee7143e370b => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:38:51.410945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:51.601581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:51.636634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3cdee-d4cf-40c8-9e53-8d35950af257 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:51.705390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:51.706236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:51.706681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:51.707160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:51.708530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] RBAC: Adding query filter params (role.id=cc467837f6074b9391a11b0e57406d12, user.id=27f6c2bbb05c432a95f0eee7143e370b, scope.system=all) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:51.708996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:51.711386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0bada-15ca-42de-a2ab-d4cccda80987 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:51.739003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 167/334] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Sep 12 23:38:51 2022] GET /identity/v3/role_assignments?user.id=27f6c2bbb05c432a95f0eee7143e370b&scope.system=all&role.id=cc467837f6074b9391a11b0e57406d12 => generated 510 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:53.809732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe211-ce53-48a4-86c7-66ed5cbc2c9b None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:53.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe211-ce53-48a4-86c7-66ed5cbc2c9b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:53.810452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe211-ce53-48a4-86c7-66ed5cbc2c9b None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:53.812135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 168/335] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:53.818251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7740-389b-4bfc-9864-cb0da39f6249 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:53.818900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7740-389b-4bfc-9864-cb0da39f6249 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:53.818900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e7740-389b-4bfc-9864-cb0da39f6249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:53.848383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7c2e7740-389b-4bfc-9864-cb0da39f6249 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:53.849639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7c2e7740-389b-4bfc-9864-cb0da39f6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:53.865583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568bbd-6d03-4443-bfd3-6e86ca84977a None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:53.866049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568bbd-6d03-4443-bfd3-6e86ca84977a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:53.866519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5568bbd-6d03-4443-bfd3-6e86ca84977a None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:53.869551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 169/336] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:53.876291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ba4b-dc1b-4c12-ae29-ab663db84f08 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:53.876772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ba4b-dc1b-4c12-ae29-ab663db84f08 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:53.877233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ba4b-dc1b-4c12-ae29-ab663db84f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:53.914852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-95f9ba4b-dc1b-4c12-ae29-ab663db84f08 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:53.916589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-95f9ba4b-dc1b-4c12-ae29-ab663db84f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:53.950972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 168/337] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:54.044320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 170/338] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:54.069886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9709a-1d31-4562-ab22-66392af497d4 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.070176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9709a-1d31-4562-ab22-66392af497d4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.070453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9709a-1d31-4562-ab22-66392af497d4 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.071939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 169/339] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:54.078830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.079473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.079931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.109867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:54.111562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:54.200804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 171/340] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:54.605524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9709a-1d31-4562-ab22-66392af497d4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:54.710415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:54.715432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:54.718816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:54.773182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9709a-1d31-4562-ab22-66392af497d4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:54.831144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:54.831968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.832128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.832402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.853953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-47d9abc5-2265-4628-bfb7-008eb2e435fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:54.859955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:54.860837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:54.861609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:54.862743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 170/341] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:38:54.863146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c933e-9b28-4371-b6ff-00e2795b0bd7 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.863599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c933e-9b28-4371-b6ff-00e2795b0bd7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.864042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c933e-9b28-4371-b6ff-00e2795b0bd7 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.865559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 171/342] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:54.872034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:54.872570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:54.872985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:54.904861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:54.906362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:54.943117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d4c1c-846b-46e3-948a-8f66590dc2f8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:54.989273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 172/343] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:38:54.990507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.000470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.001030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.001409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.001773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.027288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:55.029767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:55.030828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:38:55.031756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:55.040185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 172/344] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:38:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:55.042517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.107624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:55.178692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1511016-bbdc-4083-a12a-5b73c5d7bd66 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:55.200968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:55.251962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.252600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.252872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.254057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.254785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:55.255089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:55.256545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:55.268055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb834515-158e-49ef-93ad-53aac30dc3e3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:55.280152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 173/345] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:55.283098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.354171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.354841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.355115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.355436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.381459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:55.389510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:55.390718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:55.391548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:55.398227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 173/346] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:55.399611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.513861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa9c7e-e81b-47b7-a5d0-43022467389b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:55.535721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c4e0-d6a1-4af6-b87f-96c6a771057d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:55.539326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:55.568390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.569559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.569953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:55.570109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.570430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.571797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:55.572371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:55.574726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f3611-6d40-4863-9452-ca19c1d2cd24 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:55.591332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 174/347] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:55.592695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97583d-2816-489e-9b45-ea7d76b08e24 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.593129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97583d-2816-489e-9b45-ea7d76b08e24 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.593686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97583d-2816-489e-9b45-ea7d76b08e24 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.594606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.595270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.595554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.595821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.596184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 175/348] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:55.597575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97583d-2816-489e-9b45-ea7d76b08e24 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.715473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:55.717794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8edc1-5696-4026-8eb0-c3ace3283fab None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:55.787036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97583d-2816-489e-9b45-ea7d76b08e24 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:55.847294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:55.848272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.848272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.848543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.867776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:55.876813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:55.878604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:55.879868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 176/349] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:55.881099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.886644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 174/350] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:55.890324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab438a-1016-4111-8bce-032107d66ffc None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:55.891106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab438a-1016-4111-8bce-032107d66ffc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:55.891794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab438a-1016-4111-8bce-032107d66ffc None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:55.894898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 175/351] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:38:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:38:55.896380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab438a-1016-4111-8bce-032107d66ffc None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:55.939700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:55.956488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.057525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbab272-5363-47f2-b9bb-8b44b9c927f2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.062842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab438a-1016-4111-8bce-032107d66ffc None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.125013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.126373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.127034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.127598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.127788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.128226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.128501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.128771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.128908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:56.131377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.132060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34528aa5-c951-4a13-89bc-b2679aab53e5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:56.149081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 177/352] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:56.150943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.150943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.151245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.173191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:56.175854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:56.235922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.237425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:56.266877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.290767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 178/353] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:56.291713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:56.339661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.380819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 176/354] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:56.382059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:56.456690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dede82-3912-46eb-809d-7a2b67181af4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.524370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f1efc-cffb-4555-86d7-3971ee633705 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.537913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.538218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.538338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.538723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.539536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:56.539947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.541482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad33562-0cf3-49bb-b4cb-0aafcb44ee18 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:56.555521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 179/355] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:56.556901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.557178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.557445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.567969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.579775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:38:56.581007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:38:56.582743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.583431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.583708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.584004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.585294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.586826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:56.588842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 177/356] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:56 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:56.591482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:56.687265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 180/357] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:38:56.687265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:56.718173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.769434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b27eb-5426-4ebf-995d-971f1a892786 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.818533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.819171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.819470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.819741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.827431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d719b-60f2-4172-a08f-3b1195d3f1d4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:56.896185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:56.896780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:56.897144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:56.897616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:56.900023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.901557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:56.903568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 181/358] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:56 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:56.904189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:56.906842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:56.927872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:56.932837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.079438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.090325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fee01d-3ff6-4103-8a4e-e610443ff764 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.118081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 178/359] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:57.119210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.135887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.136521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.137271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.137550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.217164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.218615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.245235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.262938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256f557-743f-40cc-8c46-8e1917427f5c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.314921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.315825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.316320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.316804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.321457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.327287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.329941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.330662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 182/360] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:57.331550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.339425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 179/361] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:57.339425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.455758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc30ac0-d15f-4a88-902a-621d02b871df None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.478680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cf76-5d45-404d-a141-76b58fe98ec9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.516252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.517337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.518093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.518340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.534147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.535146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.535611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.536034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.537268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:57.537878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.540484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.543462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 180/362] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:57.544502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:57.547257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:57.548165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 12 23:38:57.548955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:38:57.549585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 183/363] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:57.550850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.579820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.635075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.692459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9523f-ca5e-4a4f-afb4-4369adabfa59 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.705800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a196e-2ad5-4a9b-b4d4-a19e03bd6b5e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.734949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.735699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.736024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.736338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.748747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.749399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.749668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.749932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.750051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:38:57.751131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.752646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.754361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 184/364] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:57.755141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:38:57.756286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:38:57.757886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:38:57.759563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 181/365] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:38:57.762214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.873757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.892681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da807c-8f73-4711-a9c6-017c0b52fa67 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.904635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e55cd-5b04-4184-a848-b58b69294627 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:57.922256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:57.942337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.943150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.943549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.943908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] PATH_INFO: `/v3/projects/16141215d11e48819148a2d135301f22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.945359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] RBAC: Authorizing `identity:get_project(project_id=16141215d11e48819148a2d135301f22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.946377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:57.947012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:57.947176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.947404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:57.947547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:57.948310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:57.948611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:57.949228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 185/366] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/projects/16141215d11e48819148a2d135301f22 => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:57.950104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:57.950570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:57.962465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 182/367] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:57.963592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.133401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d52b1-9f21-4096-98e6-5a20f814809f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.143292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af19f3-434d-46bb-b88c-4a6666b377f7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.180072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.180766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.181063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.181338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.182053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=ade415429d4a434eb11812b5d5fe5722) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:58.182363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.183942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.190681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 186/368] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.191622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.229401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.230036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.230305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.230572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.231308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:58.231613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.233086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.243876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 183/369] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.244786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.260185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:58.335158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:58.401084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8ec57-5cb3-4fbe-bb26-5d1e8ba53322 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.422547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ed3ec-6a25-467e-b175-7dde647dbde9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.467628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.468335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.468619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.468895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.470262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.471833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.473747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.473910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 187/370] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.474672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.480851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.482644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.483126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.484336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:58.484804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.491461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.511625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 184/371] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.513431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.632409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880b401-b6fa-48f8-a4fa-fdb30d743e62 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.657631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:58.671097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:58.693037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.694138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.694623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.695097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] PATH_INFO: `/v3/groups/caccbe2a1f124cfd96ad673465900200` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.703965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db58ddd-1353-4508-9921-2bf9a4f08db4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.707497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] RBAC: Authorizing `identity:get_group(group_id=caccbe2a1f124cfd96ad673465900200)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.710217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.713514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 188/372] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/groups/caccbe2a1f124cfd96ad673465900200 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.715022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.768014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.769391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.770372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.770820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] PATH_INFO: `/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.773048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=a795c20ded0b4cd992c9474b9ddd3ce7)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.776484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:58.779789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 185/373] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/users/a795c20ded0b4cd992c9474b9ddd3ce7 => generated 293 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:58.781439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:58.919737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b434f-91d9-4fda-90af-bb030e2a1bf2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.983818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:58.984775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:58.985232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:58.985659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:58.987002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:58.987535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:58.989605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726a26-5cf2-4bf6-92de-54e2ac227dc2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:58.990116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:59.000002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 189/374] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=cc467837f6074b9391a11b0e57406d12 => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:59.006788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:59.069856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:59.070818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:59.079250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:59.079250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:59.103873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:59.193549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:59.279257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:59.283261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:59.360690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92af4e-3430-4d73-85fb-2c50da321b22 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:59.442815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:59.443487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:59.443827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:59.444061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:59.444797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] RBAC: Adding query filter params (group.id=caccbe2a1f124cfd96ad673465900200, role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=16141215d11e48819148a2d135301f22) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:59.445107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:59.446672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:59.461971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 190/375] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:38:59 2022] GET /identity/v3/role_assignments?group.id=caccbe2a1f124cfd96ad673465900200&scope.project.id=16141215d11e48819148a2d135301f22&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 260 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:59.463425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:59.539113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 186/376] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:38:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:38:59.548277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:38:59.646982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:59.684073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a9d0b-db09-445f-8862-b668017f55b8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:59.713491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:38:59.778460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:59.779533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:59.780013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:59.780490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:59.781836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] RBAC: Adding query filter params (role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9, user.id=a795c20ded0b4cd992c9474b9ddd3ce7, scope.system=all) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:38:59.782365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:59.785178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:59.844200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 191/377] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Sep 12 23:38:59 2022] GET /identity/v3/role_assignments?user.id=a795c20ded0b4cd992c9474b9ddd3ce7&scope.system=all&role.id=b5f5ea897d5c4c0c81eb4ac24796b7c9 => generated 510 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:59.854506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d1808-8771-44f6-9685-b3a92079b95b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:38:59.909402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:38:59.910063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:38:59.910345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:38:59.910614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:38:59.912053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:38:59.913699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:38:59.916211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 187/378] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:38:59 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:38:59.922738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:00.077720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d8884-dcad-45f6-8993-320a072b8ce2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:00.128617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:00.129749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:00.130235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:00.130712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:00.132998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:00.135939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3112c-d421-4890-9d6a-da9053744085 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:00.139531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 192/379] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:38:59 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:00.145801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:00.362089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1c53-1805-4900-af05-2bde0a2595d9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:00.391670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:00.444886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:00.445842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:00.446463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:00.446937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:00.448350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=ade415429d4a434eb11812b5d5fe5722) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:00.448895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:00.451688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2b5f-0e73-4363-87db-a2c154eca9a8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:00.467892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 188/380] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:00 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=cc467837f6074b9391a11b0e57406d12 => generated 260 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:00.992444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e4bb-d281-4830-8ea6-8445d5aa94f1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:00.993258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e4bb-d281-4830-8ea6-8445d5aa94f1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:00.993739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e4bb-d281-4830-8ea6-8445d5aa94f1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:00.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 193/381] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:01.002325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d5dc-07ec-4496-981f-fb4c14b8ee3e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:01.002807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d5dc-07ec-4496-981f-fb4c14b8ee3e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:01.003285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4d5dc-07ec-4496-981f-fb4c14b8ee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:01.043363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-36b4d5dc-07ec-4496-981f-fb4c14b8ee3e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:01.048825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-36b4d5dc-07ec-4496-981f-fb4c14b8ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:01.191670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 189/382] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:02.001813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443e4bb-d281-4830-8ea6-8445d5aa94f1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:02.077787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:02.124580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bbd-91ae-4603-8d95-9159754fd005 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:02.126103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bbd-91ae-4603-8d95-9159754fd005 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:02.126842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bbd-91ae-4603-8d95-9159754fd005 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:02.130103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 190/383] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:02.139877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815818-497a-424f-9930-e6427bb09244 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:02.140770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815818-497a-424f-9930-e6427bb09244 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:02.141516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815818-497a-424f-9930-e6427bb09244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:02.165016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443e4bb-d281-4830-8ea6-8445d5aa94f1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:02.172380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2d815818-497a-424f-9930-e6427bb09244 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:02.174882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2d815818-497a-424f-9930-e6427bb09244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:02.260144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:02.260697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:02.260968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:02.261224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:02.289358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:02.296552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:02.297662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:02.298522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:02.305546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 194/384] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:39:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:02.314510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:02.321699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 191/385] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:02.545299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:02.552259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe5a26-8bf5-4932-b986-3a74ba45f5c7 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:02.607507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:02.608126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:02.608402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:02.608662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:02.609334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:02.609630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:02.611039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:02.620678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 195/386] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:02.629758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d815818-497a-424f-9930-e6427bb09244 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:02.698338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:02.796085 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d815818-497a-424f-9930-e6427bb09244 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:02.848012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:02.848960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:02.849372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:02.849783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:02.851165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:02.852854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:02.854940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 192/387] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:02 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:02.863878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:02.917158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:03.135700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d38cd-e40f-464b-b064-7bc3c47306e8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:03.152056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:03.175676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303992c-1083-4c85-80d0-45c9c63ccd03 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:03.203319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:03.204879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.205678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.206430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.208683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:03.211600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:03.215277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 196/388] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:02 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:03.237418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:03.256576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:03.257240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.257504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.257935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.271194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-de6407f1-8136-4e10-814b-50dd78f532aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:03.276626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:03.278501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:03.287148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 193/389] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:39:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:03.289257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e010-9948-497e-8bac-530e889a74c3 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.289780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e010-9948-497e-8bac-530e889a74c3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.290287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e010-9948-497e-8bac-530e889a74c3 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.293189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 194/390] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:03.294580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e010-9948-497e-8bac-530e889a74c3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:03.318044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:03.389989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3743e-4490-404a-997c-e31d69d844cd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:03.450572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:03.452521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.454001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.455054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/groups/78d6b031be3542b6b1470cee3e1660e5/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.460077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, group_id=78d6b031be3542b6b1470cee3e1660e5, role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:03.464826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa1617-a6df-4208-b341-708b6a01bd79 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:03.476482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e010-9948-497e-8bac-530e889a74c3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:03.499133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 197/391] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Sep 12 23:39:03 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/groups/78d6b031be3542b6b1470cee3e1660e5/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:39:03.502747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.503683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.504423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.539970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:03.539970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:03.556581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:03.557646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.558143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.558620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.559868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:03.560398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:03.563095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:03.598971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 195/392] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:03.607473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:03.628043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 198/393] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:03.653400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:03.761777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d3c68-e526-41c6-9f90-ddc2822d25a0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:03.805962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:03.806913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.807381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.807794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.809460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:03.811109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7340e-64e7-445d-977e-ead061d4f862 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:03.813477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 196/394] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:03 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:03.820408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:03.878237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:03.973282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ab04-2a6d-4bea-9f1e-803e91a2e72f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.973470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ab04-2a6d-4bea-9f1e-803e91a2e72f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.973742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4ab04-2a6d-4bea-9f1e-803e91a2e72f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:03.975423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 197/395] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:03.981020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:03.981298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:03.981574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.020520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:04.021889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:04.027352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a296ed-ef01-49d6-9eca-bd946164556c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:04.096239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:04.097278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.097724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.098191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.100261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:04.103060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc174d4-d791-4c07-9a2c-abef76ad2087 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:04.107409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 199/396] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:03 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:04.108243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bb159-febf-4b30-be99-aa2eb4ffce94 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.108725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bb159-febf-4b30-be99-aa2eb4ffce94 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.109193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bb159-febf-4b30-be99-aa2eb4ffce94 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.115262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 200/397] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:04.115262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bb159-febf-4b30-be99-aa2eb4ffce94 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.121082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 198/398] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:04.122321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.147044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:04.269182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:04.287663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bb159-febf-4b30-be99-aa2eb4ffce94 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:04.294959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0039-230c-4163-9b5a-507a003fd474 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:04.327850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:04.328854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.329317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.329805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.344504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:04.356654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-77a1b8b4-89e0-4b68-8056-cf2c1d05d093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:04.357601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:04.358526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:04.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:04.359915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 201/399] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:04.365624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.365624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.365624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.366370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.366939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.367423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/groups/78d6b031be3542b6b1470cee3e1660e5/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.370131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, group_id=78d6b031be3542b6b1470cee3e1660e5, role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:04.371868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:04.399322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:04.399322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:04.400041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 199/400] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Sep 12 23:39:04 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/groups/78d6b031be3542b6b1470cee3e1660e5/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:04.401256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.516023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:04.542345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 202/401] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:04.550435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.562877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7463d-2ce0-454f-8574-e1549b86f7d4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:04.672411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:04.674066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.675399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.676482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.678036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:04.678862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:04.688666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:04.721501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 200/402] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:39:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:04.729113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:04.734499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.781967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed5a4-e098-42f8-910d-0c847ee74bcb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:04.859714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:04.861254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:04.861541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:04.861805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:04.878140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:04.884405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:04.886452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:04.888416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:04.892713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 203/403] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:04.892713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:04.974198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b915215-0088-4a59-a000-9dca8612349e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.056573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.057912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.058518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.059113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.059300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ffe5-39f4-4073-99a7-9e9396c3310a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.061627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.072660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.072660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 201/404] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:04 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.074758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.100286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.110633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.111179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.111494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.111762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.122075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.133581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:05.144866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:05.146629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:05.148254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:05.152984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 204/405] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:05.154842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.260025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382bc90-49c0-494f-9bd6-fd4e10eb1e8e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.323728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b09b3-a916-49cc-850e-2c4869f40398 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.355781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.358615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.359497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.359793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.360525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:05.360908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.363978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.377013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.378192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 202/406] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.378914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.379541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.380284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.381003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.383477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.386768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.390306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 205/407] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.391532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.468020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.507245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.518027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce73da2-0122-4d9d-9d79-4c3113ff1dce None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.547368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fafa-8be6-4066-838e-8fe7d1bd83d7 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.576274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.578214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.578725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.579444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.580802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:05.581543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.583864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.588294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.589056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.589378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.589717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.591074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.592803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.598924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 206/408] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.598924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.601548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 203/409] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.603039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.717603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc6c44-fcb9-4693-aaf9-948f763e391c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.727269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d0ed5-e804-496a-9cac-c4b8e6bda175 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:05.750352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.757278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.758059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.758657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.758657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.760024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.761789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.768180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 204/410] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:05.768180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.777712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:05.778613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:05.778987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:05.779386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/groups/78d6b031be3542b6b1470cee3e1660e5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:05.782191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f42dee1d107e42cf88a3999f3871ae0c, group_id=78d6b031be3542b6b1470cee3e1660e5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:05.783829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:05.787678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:05.806386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 207/411] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Sep 12 23:39:05 2022] PUT /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c/groups/78d6b031be3542b6b1470cee3e1660e5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:05.807283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:05.954655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70420590-a186-4916-bfdf-45f56bb11d9a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:06.007973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:06.008684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:06.008972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:06.009248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] PATH_INFO: `/v3/groups/caccbe2a1f124cfd96ad673465900200` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:06.012536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] RBAC: Authorizing `identity:get_group(group_id=caccbe2a1f124cfd96ad673465900200)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:06.014274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:06.016548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 205/412] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:05 2022] GET /identity/v3/groups/caccbe2a1f124cfd96ad673465900200 => generated 229 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:06.024766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db4703-d9eb-4456-8bd5-242647ddf6c4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:06.026801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:06.120195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:06.145653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:06.147607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:06.148237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:06.148717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722/groups/78d6b031be3542b6b1470cee3e1660e5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:06.152154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] RBAC: Authorizing `identity:create_grant(project_id=ade415429d4a434eb11812b5d5fe5722, group_id=78d6b031be3542b6b1470cee3e1660e5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:06.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:06.177197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac82ae-56f8-477f-9962-7112d8540c46 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:06.205737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 208/413] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Sep 12 23:39:05 2022] PUT /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722/groups/78d6b031be3542b6b1470cee3e1660e5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:06.257411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:06.258161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:06.258441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:06.258709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] PATH_INFO: `/v3/projects/16141215d11e48819148a2d135301f22` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:06.260064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] RBAC: Authorizing `identity:get_project(project_id=16141215d11e48819148a2d135301f22)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:06.261570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7c24f-c932-4c8c-9771-c3c8ab24359d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:06.263387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 206/414] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:06 2022] GET /identity/v3/projects/16141215d11e48819148a2d135301f22 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:06.269788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:06.394688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:06.473396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee7f0-aa27-460f-a44f-d22936730211 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:06.522869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:06.523501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:06.523778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:06.524086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] PATH_INFO: `/v3/projects/16141215d11e48819148a2d135301f22/groups/caccbe2a1f124cfd96ad673465900200/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:06.526090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] RBAC: Authorizing `identity:create_grant(project_id=16141215d11e48819148a2d135301f22, group_id=caccbe2a1f124cfd96ad673465900200, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:06.527650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a544a73-07e8-4a5e-9ed9-9e7d8f672533 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:06.563715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 209/415] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Sep 12 23:39:06 2022] PUT /identity/v3/projects/16141215d11e48819148a2d135301f22/groups/caccbe2a1f124cfd96ad673465900200/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:07.019542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516d454-b33b-4059-a7ad-048627353a62 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:07.021094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516d454-b33b-4059-a7ad-048627353a62 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:07.021861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516d454-b33b-4059-a7ad-048627353a62 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:07.025206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 207/416] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:07.031664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddadd4-a9e3-457a-9331-628fa93af6b0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:07.032114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddadd4-a9e3-457a-9331-628fa93af6b0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:07.032578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddadd4-a9e3-457a-9331-628fa93af6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:07.069744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-faddadd4-a9e3-457a-9331-628fa93af6b0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:07.071838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-faddadd4-a9e3-457a-9331-628fa93af6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:07.246034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 210/417] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:07.571925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516d454-b33b-4059-a7ad-048627353a62 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:07.814852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516d454-b33b-4059-a7ad-048627353a62 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:07.904271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:07.905802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:07.906534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:07.907287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:07.923337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:07.940990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:07.949382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22c4a-cfae-44af-964e-8b495dad8fcd None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:07.950176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22c4a-cfae-44af-964e-8b495dad8fcd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:07.950176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22c4a-cfae-44af-964e-8b495dad8fcd None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:07.951673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 211/418] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:07.953906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:07.955707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:07.957237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:07.958377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:07.958377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:07.958377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:07.962554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 208/419] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:39:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:07.970220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:07.978895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:07.980159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:08.089700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:08.198650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 212/420] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:08.255411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a65c8-5947-448b-9471-7f6a10f548d5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:08.340646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:08.342136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:08.343207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:08.343975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:08.345531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:08.346321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:08.349405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:08.366813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 209/421] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:08.373311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:08.526036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:08.542893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847bb2b-3e75-427c-bad3-b4af0380a5c9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:08.583291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:08.583291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:08.583291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:08.583291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:08.630897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:08.659798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:08.661330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:08.763966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:08.777485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 213/422] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:08.783243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:08.895924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:08.924037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2014d-2e0c-49cb-964c-262fbefb6fbc None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:08.948495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907660-f010-43ce-8bee-3ae1cd26a40c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:08.983187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:08.985473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:08.986260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:08.987017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:08.989704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:08.993020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:08.997377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 214/423] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:08 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:09.003861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:09.107757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:09.109434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.110182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.110933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.153450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:09.171380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:09.172969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:09.174340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 210/424] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:39:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 649 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:09.188974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:09.271121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03315-ec37-408c-a8dd-d297b51e070e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:09.377498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:09.378499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.378986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.379506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.384537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:09.387389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:09.396041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 215/425] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:09 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:09.402499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:09.421496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:09.503232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8804ab-2ab0-426c-a73a-489267f2aa57 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:09.538526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd04a6f-1127-40a4-a03c-9b379f499e6b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:09.539200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:09.565009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:09.565647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.565896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.566166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.566841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:09.567151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:09.568588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e81e1-e1da-4d43-b6f4-9302a46a1f2a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:09.582711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 211/426] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:09.582711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a3592-f330-4c03-b34f-7e89c069dc74 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.582711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a3592-f330-4c03-b34f-7e89c069dc74 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.582711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a3592-f330-4c03-b34f-7e89c069dc74 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.586237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 212/427] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:09.587151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a3592-f330-4c03-b34f-7e89c069dc74 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:09.594552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:09.595583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.596088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.596567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.598260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=ade415429d4a434eb11812b5d5fe5722) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:09.598780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:09.601038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ec21f-869d-4510-897b-f6c1110e4cb9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:09.610344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 216/428] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:09 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:09.612014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a14a5-d1e6-4594-9060-7fe635e32d46 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.612509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a14a5-d1e6-4594-9060-7fe635e32d46 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.613016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a14a5-d1e6-4594-9060-7fe635e32d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.653840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0d3a14a5-d1e6-4594-9060-7fe635e32d46 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:09.655497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0d3a14a5-d1e6-4594-9060-7fe635e32d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:09.737328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 217/429] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:09.739091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dcd2b-c409-4300-b808-6a180dc3c604 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.739598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dcd2b-c409-4300-b808-6a180dc3c604 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.740055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dcd2b-c409-4300-b808-6a180dc3c604 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.741864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 218/430] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:09.753288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.754017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.754474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.768172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:09.793794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:09.795481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:09.800017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a3592-f330-4c03-b34f-7e89c069dc74 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:09.881885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:09.882728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:09.883123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:09.883458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:09.913393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 219/431] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:09.953420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:09.981132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:09.982935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.020347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.106735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 213/432] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:10.113040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.193728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632d158-7a84-444d-b863-e07b308320a4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.250896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.251701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.252002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.252235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.254856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.255269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.269136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:10.274293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:10.275332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 220/433] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:10.280267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.288229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc31c2-c21b-49f8-a0b3-7cf03a9632bd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.329974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.330622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.330902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.331171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.332536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:10.334134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.341404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 214/434] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:10.341404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.399593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f3db-d7c7-49f4-bd1d-59558d1d84b9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.437734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.438504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.438907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.439304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.450529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7e983-cb4d-47d3-876e-2d0922203510 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.452308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-08bde473-ac7a-42d8-940e-b67d59724ef8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:10.457571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:10.458555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:10.459454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 221/435] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:10.463285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d20c4-312a-48b6-bb2a-5fa560f38c02 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.463285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d20c4-312a-48b6-bb2a-5fa560f38c02 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.463285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d20c4-312a-48b6-bb2a-5fa560f38c02 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.466576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 222/436] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:10.466576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d20c4-312a-48b6-bb2a-5fa560f38c02 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.494496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.494496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.495594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.495594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.495795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:10.495982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:10.497370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.506608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 215/437] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:10.507480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.530953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.534946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.591891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d20c4-312a-48b6-bb2a-5fa560f38c02 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.635415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.636059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.636336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.636599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.637752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:10.639209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c458e510-9b2f-4d21-8e74-5ec053b78eca None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.640847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 223/438] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:10.642037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.642269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.642503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5e83-4245-404b-b417-75e7e6e599ca None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.642671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.662556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:10.663768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:10.680369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.681005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.681363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.681526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.682207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] RBAC: Adding query filter params (name=member) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:10.682508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:10.683958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.693619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 216/439] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:10.694440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.738479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 224/440] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:10.739287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:10.792205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.831919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:10.835603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a2ca-e9e0-4e96-804e-9e7fdd058aeb None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.888636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.889348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.889704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.890153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.894762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af237-48b9-4d2f-995a-7e19457d2237 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:10.941938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:10.942760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:10.943116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:10.943519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:10.944467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:10.944863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:10.946549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:10.963267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 225/441] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:10.963267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.000190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.002510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.102354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.111418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb550a1d-9f03-4622-a5b9-7bd685d64bc8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.139489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.145443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 217/442] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:11.155376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.174774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.175465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.175882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.176034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.268173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.270572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.292162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3902b71-b091-473c-8ed3-62bc5c62142f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.344482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.345134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.345411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.345673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] PATH_INFO: `/v3/projects/ade415429d4a434eb11812b5d5fe5722` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.346840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] RBAC: Authorizing `identity:get_project(project_id=ade415429d4a434eb11812b5d5fe5722)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.348359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.350862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 218/443] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/projects/ade415429d4a434eb11812b5d5fe5722 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.351670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.397713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 226/444] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:11.398620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.399203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.420379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.473126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5b1b-1848-46fa-976f-0d3ccd03f529 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.512743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.513440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.513675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.514464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.528453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:11.532739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbd5a-9ccd-4e99-9f1e-23abb28e8243 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:11.533574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:11.534457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 219/445] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:11.540790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.571302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.572266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.572731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.573167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.574612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.576324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.577988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 227/446] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.578828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.656564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.660351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac1a9d-82ee-4e9d-9662-b0a0f7ae3fa7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.682674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.707080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2f16-7d28-4cb8-8e91-4a32c9040956 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.708008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.708741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.709088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.709532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] PATH_INFO: `/v3/projects/f42dee1d107e42cf88a3999f3871ae0c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.711340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] RBAC: Authorizing `identity:get_project(project_id=f42dee1d107e42cf88a3999f3871ae0c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.713006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.719513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 220/447] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/projects/f42dee1d107e42cf88a3999f3871ae0c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.719513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.756462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.757075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.757358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.757624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.758451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:11.758750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.760323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.771879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 228/448] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.772742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.834740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87efd0a-6c20-4e8c-93a7-3122dbef395a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.870992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.871624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.871897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.872167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.872895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=ade415429d4a434eb11812b5d5fe5722) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:11.873194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.874620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.879766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120177-37aa-4c84-895e-e7af0322129e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:11.880619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 221/449] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=ade415429d4a434eb11812b5d5fe5722&role.id=cc467837f6074b9391a11b0e57406d12 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.881387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.922400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:11.940143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:11.941154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:11.941154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:11.941360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] PATH_INFO: `/v3/groups/78d6b031be3542b6b1470cee3e1660e5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:11.942583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] RBAC: Authorizing `identity:get_group(group_id=78d6b031be3542b6b1470cee3e1660e5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:11.944123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:11.950984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 229/450] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/groups/78d6b031be3542b6b1470cee3e1660e5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:11.952791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:11.982831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:12.029363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b43b-19e1-43fc-994f-c40e35357c4e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:12.088705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:12.089915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:12.089915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:12.089915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:12.115650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483cd3-732b-4340-9e69-95f9e5fb86cb None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:12.156534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:12.157373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:12.157776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:12.158229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:12.159554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] RBAC: Adding query filter params (group.id=78d6b031be3542b6b1470cee3e1660e5, role.id=cc467837f6074b9391a11b0e57406d12, scope.project.id=f42dee1d107e42cf88a3999f3871ae0c) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:12.159871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:12.161351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:12.168338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 230/451] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/role_assignments?group.id=78d6b031be3542b6b1470cee3e1660e5&scope.project.id=f42dee1d107e42cf88a3999f3871ae0c&role.id=cc467837f6074b9391a11b0e57406d12 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:12.191374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:12.192926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:12.229640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:12.320967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 222/452] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:12.328033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:12.446007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:12.477236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587de7f6-f608-41fa-9af7-34a2fef63a4d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:12.528522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:12.529283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:12.529648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:12.530107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] PATH_INFO: `/v3/projects/16141215d11e48819148a2d135301f22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:12.532138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] RBAC: Authorizing `identity:get_project(project_id=16141215d11e48819148a2d135301f22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:12.534532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:12.536798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 231/453] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:39:12 2022] GET /identity/v3/projects/16141215d11e48819148a2d135301f22 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:12.544946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:12.705923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac6e5-5fc4-4be9-a12c-9b92ba452872 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:12.741326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:12.747505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:12.748163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:12.748432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:12.748700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] PATH_INFO: `/v3/groups/caccbe2a1f124cfd96ad673465900200` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:12.749910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] RBAC: Authorizing `identity:get_group(group_id=caccbe2a1f124cfd96ad673465900200)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:12.751398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a29f0a-7038-4abf-8b8e-3df55dae85e6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:12.753037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 223/454] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Sep 12 23:39:12 2022] GET /identity/v3/groups/caccbe2a1f124cfd96ad673465900200 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:12.760200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:12.894008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8234750-85e2-4307-b863-58891be3f361 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:12.943676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:12.944444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:12.944801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:12.945151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:12.946089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] RBAC: Adding query filter params (group.id=caccbe2a1f124cfd96ad673465900200, role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=16141215d11e48819148a2d135301f22) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:12.946488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:12.948161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54962f0-518f-4c1b-94d4-04dc9a8b6da4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:12.957087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 232/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Sep 12 23:39:12 2022] GET /identity/v3/role_assignments?group.id=caccbe2a1f124cfd96ad673465900200&scope.project.id=16141215d11e48819148a2d135301f22&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:15.777181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fefe0c-0267-4c4f-9175-b9b54f798fa8 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:15.777709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fefe0c-0267-4c4f-9175-b9b54f798fa8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:15.777999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fefe0c-0267-4c4f-9175-b9b54f798fa8 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:15.780882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 224/456] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:15.790133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080b2d-ebe4-40ee-9ca0-d507444868b0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:15.791562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080b2d-ebe4-40ee-9ca0-d507444868b0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:15.792276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5080b2d-ebe4-40ee-9ca0-d507444868b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:15.827611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a5080b2d-ebe4-40ee-9ca0-d507444868b0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:15.829374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a5080b2d-ebe4-40ee-9ca0-d507444868b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:15.837214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:15.948675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 233/457] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:16.393390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab42c1-ad23-4580-b3fc-73dd93cb0b5f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:16.393571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab42c1-ad23-4580-b3fc-73dd93cb0b5f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:16.393844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab42c1-ad23-4580-b3fc-73dd93cb0b5f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:16.395619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 225/458] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:16.401994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:16.402332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:16.402550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:16.427365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:16.429013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:16.455440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab42c1-ad23-4580-b3fc-73dd93cb0b5f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:16.565545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 234/459] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:16.672644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab42c1-ad23-4580-b3fc-73dd93cb0b5f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:16.722598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:16.723295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:16.723577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:16.723858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:16.736203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:16.737635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:16.765684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:16.766645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:16.767508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:16.767508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:16.767508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:16.769384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 226/460] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:16.779363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:16.948194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:16.993069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6370f15-daad-4785-88bc-951ec69f9e76 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:17.045112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:17.045762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.046042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.046316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.047034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:17.047339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:17.048821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22d10e-3d8e-4c46-9a51-a3a5866c68d9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:17.070674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 235/461] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:17.078213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:17.114860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ab333-2cbb-4227-a375-3677ff32929a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.115260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ab333-2cbb-4227-a375-3677ff32929a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.115448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ab333-2cbb-4227-a375-3677ff32929a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.117878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 236/462] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:17.124579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.124923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.125141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.163247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:17.166251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:17.187840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:17.269888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aeecd-055c-450f-ad12-714805a5a0c4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:17.302304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 237/463] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:17.304945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:17.315506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:17.316130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.316404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.316668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.317323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:17.318819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192e30f-afc5-43bb-a5fc-e1a4d44e4586 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:17.361377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 227/464] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:17 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:17.461702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:17.577712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b97f29-ccf1-4db0-8568-434ca3425dea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:17.646719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:17.647862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.648378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.649303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.670983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:17.673809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:17.694116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e119f-fd90-42af-8d1f-5057da778ae0 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.694405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e119f-fd90-42af-8d1f-5057da778ae0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.694675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e119f-fd90-42af-8d1f-5057da778ae0 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.697262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 228/465] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:17.702540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:17.702855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:17.703103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:17.705323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:17.706232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:17.707062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:17.707062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:17.707062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:17.709152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 238/466] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:17.720616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:17.727142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:17.729794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:17.831011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 229/467] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:17.831956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:17.885692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:17.917850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca06e-6d4b-41ac-bdbe-cfa4133cf1b6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:17.958750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:18.003437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.004341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.004780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.005307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.006219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:18.006735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.008481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115d9d-21bf-49f5-a0a3-b645282423b1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.008693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f72ce-eed4-4577-98e4-314cd791affc None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.030894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 239/468] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:18.030894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c21d-4d3f-4aeb-9659-30d7764796e2 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.030894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c21d-4d3f-4aeb-9659-30d7764796e2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.030894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c21d-4d3f-4aeb-9659-30d7764796e2 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.030894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 240/469] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:18.032570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c21d-4d3f-4aeb-9659-30d7764796e2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.098744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.101221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.101584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.101854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.132482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.136067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3adf0d0b-7b0c-4205-ad31-3e20f95e3038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:18.167558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:18.168569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:18.169400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:18.169400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:18.169400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:18.169580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 230/470] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:18.171662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.172126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.172575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.177820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c21d-4d3f-4aeb-9659-30d7764796e2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.203645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:18.205738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:18.220262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.220898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.221171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.221442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.222127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.223778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.255268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 241/471] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:18 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:18.255268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.307141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:18.342028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 231/472] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:18.343588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.383318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:18.487824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21b597-6af3-4499-abbb-d53539b8d868 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.554014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa10d4-4b6b-4f11-8288-899881914380 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.574889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.576469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.576953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.577986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.601898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.604133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.606600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.608685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.609282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.609856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.611135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:18.611821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.615550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:18.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:18.625183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:18.626008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:18.626008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:18.626008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:18.628206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 242/473] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:18.636529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.641777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 232/474] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:18.650688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.776750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:18.800714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5854e-2a85-4783-9324-8ebe4d9b31b6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.825560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:18.872301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de648b7-6a07-42cf-8ca8-22dd5948f765 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:18.879162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.880181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.880623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.881052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.882105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.884743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.931677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 233/475] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:18 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:18.933251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:18.933374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:18.934116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:18.934402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:18.934673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:18.935529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:18.935823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:18.937521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:18.953557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 243/476] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:18.964203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:19.141821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25de08-0aea-458d-a5b2-0ffc65df2730 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:19.203675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:19.204223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:19.204504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:19.204773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:19.205492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:19.206865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:19.214003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:19.222154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.sql.core [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:39:19.222154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 23:39:19.222154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [parameters: {'id': 'e3af91737bc345d1af995e660c9d2d9b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 23:39:19.222154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87847) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:39:19.229288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e406e3-171f-4551-b1ca-2c926c6a5c7a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result.read() Sep 12 23:39:19.260753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return ref.to_dict() Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:39:19.261697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application yield resource Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise self.value Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application session.commit() Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:39:19.262578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:39:19.263457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:39:19.264319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result.read() Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:39:19.265121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [parameters: {'id': 'e3af91737bc345d1af995e660c9d2d9b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:19.267305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 12 23:39:19.275623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.277639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 234/477] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Sep 12 23:39:18 2022] POST /identity/v3/roles => generated 144 bytes in 331 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:19.277639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:19.304444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:19.305467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:19.305923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:19.306365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:19.307258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:19.309044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:19.327354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:19.361648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 244/478] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:18 2022] POST /identity/v3/users => generated 249 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:19.462832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2166-61d0-4a4b-a251-6f138a71755c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:19.514938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:19.516023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:19.516532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:19.517012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:19.538106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-86c55399-6b44-42e8-b463-054b181ad7c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:19.545694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:19.546632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 12 23:39:19.551921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:19.554997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 235/479] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:39:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:19.562222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:19.793076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee8d14-1bbc-4db5-8e82-3d967c784608 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:19.851313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:19.852281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:19.852690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:19.853169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:19.854309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:19.854758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:19.857082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b255466-6e47-4539-b3ac-352bcff45883 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:19.871736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 245/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:20.635245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d67f8-45d3-4d8f-83b4-320ea135e4cc None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:20.636926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d67f8-45d3-4d8f-83b4-320ea135e4cc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:20.637686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d67f8-45d3-4d8f-83b4-320ea135e4cc None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:20.640482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 236/481] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:20.650272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:20.652108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:20.652766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:20.695296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:20.697713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:20.809592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:20.835151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 246/482] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:21.161104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d67f8-45d3-4d8f-83b4-320ea135e4cc None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:21.266800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:21.324827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d67f8-45d3-4d8f-83b4-320ea135e4cc None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:21.380839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:21.381423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:21.381702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:21.381976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:21.403037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-681a9ef0-20cd-4c74-a844-7b2fc8721048 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:21.408495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:21.409392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:21.410626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 237/483] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:21.418720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:21.531337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca45d6-7def-4a6a-9e59-421267dc4b8f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:21.531819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca45d6-7def-4a6a-9e59-421267dc4b8f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:21.532604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca45d6-7def-4a6a-9e59-421267dc4b8f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:21.535731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 238/484] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:21.542245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:21.542863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:21.543172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:21.594797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:21.597094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:21.685805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f9f3-d97a-4776-998c-b4a007daf0bc None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:21.738754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 239/485] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:21.768367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:21.769448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:21.769914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:21.770455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:21.771722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:21.772248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:21.774996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:21.792067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 247/486] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:21.798356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:21.871904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:21.947031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6854602-4dcf-437e-8b0d-3509173f742a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.000128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.001243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.001781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.002241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.014583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:22.016623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:22.024156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:22.034684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:22.035641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:22.036483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:22.036483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:22.036483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:22.042607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 240/487] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:22.045714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.152360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:22.226838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa431ff-986f-4da2-8a19-0f21110c25d9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.229498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:22.262577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff44cb1-6ed0-4ca0-949a-edae7a6e7c2f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.302540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.304034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.304613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.305078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.322043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:22.328785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:22.329751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:22.330532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:22.332557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.333533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.334016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.334478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.335462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:22.336010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:22.337709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:22.340923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 248/488] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:22.346857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.365123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 241/489] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:22.384727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.529304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737148-e2a4-495d-8a63-d6892ea94e40 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.545359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:22.565515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a8920-1058-497d-8869-8b922c28fb22 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.585219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.586395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.586683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.586962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.587695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:22.588005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:22.589438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f66507-3125-4856-ab12-51f192d9cc09 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:22.607269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 249/490] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:22.608992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2360bec-9aee-407f-b68d-d18a7198e11a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.609271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2360bec-9aee-407f-b68d-d18a7198e11a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.609550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2360bec-9aee-407f-b68d-d18a7198e11a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.611154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 250/491] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:22.611922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2360bec-9aee-407f-b68d-d18a7198e11a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.612656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.613306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.613672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.613944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.626665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:22.629615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-fd137cad-efd0-440e-b85d-66acde8912c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:22.653593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:22.654578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:22.656023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 242/492] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:22.659414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.659738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.660013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.686520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:22.687930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:22.712880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:22.752565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2360bec-9aee-407f-b68d-d18a7198e11a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:22.796537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:22.802785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 243/493] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:22.802785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.804392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:22.804918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:22.805334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:22.828800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:22.832327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:22.876647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:22.877622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:22.878467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:22.878467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:22.878467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:22.882063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 251/494] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:22.886689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:22.960645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.030666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566edfae-c536-4625-b497-dba2b2a11b57 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.075308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcffc-2825-4e87-9065-67953e6fd5f4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.080628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.081300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.081544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.081801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.082539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:23.082836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:23.084309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6a15b-13b7-4634-be36-465222b8cfc2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:23.129713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.134649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 244/495] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:23.134649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c495ce-91f5-4334-9a13-721a38ce63d8 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.134649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c495ce-91f5-4334-9a13-721a38ce63d8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.134649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c495ce-91f5-4334-9a13-721a38ce63d8 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.134649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 245/496] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:23.136065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.136561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.137018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.137885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:23.138143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c495ce-91f5-4334-9a13-721a38ce63d8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.138701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:23.140338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90aee-3fa4-4d30-a90a-297fadf5bfea None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:23.154922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.160998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 252/497] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:23.162114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.162388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.162655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.183017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:23.184380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:23.261770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 253/498] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:23.262901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.282743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c495ce-91f5-4334-9a13-721a38ce63d8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.286865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.335762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.337061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.337656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.338159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.396034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8c492-5de0-4910-a475-7d34ae30d30d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.436204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.437012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.437493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.437760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.454092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:23.459485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:23.461288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 254/499] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:23.462751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.467173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:23.468730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fb130-b9d6-4113-9d74-41f6b087c0a2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:23.489555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.560967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.592467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0cabb-d555-42d3-8a5c-2f0d9862e285 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.629612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.630359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.630630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.630892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.631145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 246/500] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:23.632588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d474-9b09-4df6-988a-e612d23559cd None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.632874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d474-9b09-4df6-988a-e612d23559cd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.633183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d474-9b09-4df6-988a-e612d23559cd None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.634530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 247/501] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:23.635495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62d474-9b09-4df6-988a-e612d23559cd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.647772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:23.650816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:23.668845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:23.669999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:23.670966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 255/502] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:23.671677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.755809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62d474-9b09-4df6-988a-e612d23559cd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.767475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.793903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.794655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.795060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.795452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.796297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:23.796753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:23.798328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:23.805345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be665c8f-56c2-4295-96ed-094c9aa40b41 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.811148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 248/503] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:23.812753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.858534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.859084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.859438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.859675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.861832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:23.871963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-213451e5-5763-4c90-b323-b651fe6a3da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:23.877470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:23.878513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:23.879711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 256/504] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:23.880870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.880870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.880870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:23.904389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:23.905644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:23.950751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505660ba-aad4-492a-a648-52f14aeeb133 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:23.991921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 257/505] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:23.992829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:23.996031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:23.996686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:23.996958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:23.997223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.022114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.024278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:24.045110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:24.046120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:24.046942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:24.046942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.046942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:24.051173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 249/506] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:24.052625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.083548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.126090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d359ed-8ea4-4d7e-86a8-16c830d8545c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.168151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.171664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e1456-b7b5-412b-8580-1be889c9ac09 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.184301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.185036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.185313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.185580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.186329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:24.186654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.188146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.213108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.214039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.214504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.214943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.233236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 258/507] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:24.233236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.233488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.233488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:24.253211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:24.254289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:24.255334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:24.255334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.255334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:24.258874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 250/508] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:24.259840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.338774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2699a01-e032-4143-a82b-d3b7d9f3e06c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.344786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.376275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.376809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.377084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.377354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.378035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:24.378340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.379807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.382470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a79e39-1619-4ebd-a52a-23ecc59570cf None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.388612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 259/509] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:24.389435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.422669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.423289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.423564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.423836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.424488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:24.424784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.426190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.439878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 251/510] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:24.440746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.456404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.515012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d62b6-84b9-4545-9fc0-e49483ddc92c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.553514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.554145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.554471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.554747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.562416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c43d90-edbf-4e72-860d-bcd2f833db79 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.604747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.605439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.605717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.605988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.609346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.619275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.620884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.631893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.633416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:24.637322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:24.638434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:24.640634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:24.640634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.640634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:24.640634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 252/511] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:24.640634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.706307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.748623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f7b7-60a2-43e7-8fdb-39bad836929d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.785483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.786321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.786721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.787146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.787964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:24.788118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 260/512] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:24.788462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.788969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.790289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.805461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 253/513] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:24.806169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.871461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.902683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98239c81-c670-4127-9b8f-37796a300c25 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.940392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.940788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf64bc4-2718-4121-9039-e206afdd7f23 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:24.941042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.941317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:24.941582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:24.952830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:24.954335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:24.957212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:24.970848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:24.972947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:24.974810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:24.974810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:24.974810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:24.974810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 261/514] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:24.974810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:24.997982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:24.999047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:24.999599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.000089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.012450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.014953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:25.036932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:25.037855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:25.038692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:25.039093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 254/515] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:25.040421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.090259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45111-ac5a-45a2-8239-be14a0376fd0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.126164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.126787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.127059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.127367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.128027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:25.128334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.129747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.148698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 262/516] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:25.150048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.165810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbf67-7cfb-4513-a1c7-fa90a86add32 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.175092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.205107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.205848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.206190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.206524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.222046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.223997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.229536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:25.246282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:25.247308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:25.248120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:25.248120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:25.248120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:25.248120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 255/517] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:25.249550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.284715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6535-5cd3-4d6b-8853-e8ec4d5ef26c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.336872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.337682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.338063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.338443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.346350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.347894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:25.360636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:25.361624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:25.362500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 263/518] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:25.363072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.384874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c579-de7d-4334-b6f0-bf9fba876667 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.422748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.423434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.423713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.423981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.424638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:25.424939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.426360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.439666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 256/519] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:25.440512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.473079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.478690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f29653-3a8b-4f89-a0e9-9d4b51042225 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.497206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.518900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.519627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.519996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.520337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.521125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:25.521489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.523109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.551892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 264/520] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:25.551892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.568790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701383f1-83fc-40f8-86e1-af7c5318d948 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.608575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.609272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.609542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.609819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.611579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.613106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.643494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 257/521] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:25 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:25.644509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.689394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7522471-4202-4067-b4f1-1ef5e2dc61c9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.736976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.737797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.738210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.738550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.739394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:25.739798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.741402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.758653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.760616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 265/522] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:25.761870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.796827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916037a9-9d2c-4ea8-a6eb-376a8a2e7756 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.810309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:25.852245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.853150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.853574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.854012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.855376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:25.855681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.857218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.885205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 258/523] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:25.885205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:25.900127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5464c-63e3-48e8-b426-f6092da83ff6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:25.957889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:25.958547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:25.958547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:25.959482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:25.973681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:25.975259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:25.990305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:25.992587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 266/524] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:25.995026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.032919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80736d53-a539-482a-af25-c874fb7d8bae None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.084291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.085075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.085537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.085972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.086897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:26.087368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.089018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.091543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.103275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 259/525] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:26.103275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.146841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd6dab-da2e-43a0-9623-7393106f7c09 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.151688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.202573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.203266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.203606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.203883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.214694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.216635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:26.234747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:26.235789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:26.236644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:26.237048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 267/526] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:26.237729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.327662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff920525-22d0-4fae-8642-c1dbeb1cbdc3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.383083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.383759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.384040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.384312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.401718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5996f6-3871-451c-b64a-a94897a2d0c1 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.455974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.457303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.458239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.458992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.460319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:26.460912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.463413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.468833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.473108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.474762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.497039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 268/527] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Sep 12 23:39:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:26.497039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.514635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.610611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610c0ac-c0f8-46b6-ad6b-485561e7a49b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.615666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 260/528] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:26.616969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.667721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.669010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.669209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.669669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.670928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:26.671677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.676326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.698468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 269/529] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:26.704205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:26.744650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.751051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c3de-88c6-442f-8b26-754bf968fc42 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:26.796978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:26.798032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:26.798500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:26.798973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:26.896154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:26.897753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:26.900247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:26.962113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0ced-559f-4e32-8750-fe8bcc1fd6ea None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.013462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.014179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.014535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.015115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.026749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.028464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.039530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.045462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 261/530] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:27.046631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:27.050178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:27.051324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:27.052339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:27.052339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:27.052339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:27.059777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 270/531] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:27.060829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.182037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784197-199e-4cfd-af03-7f64f4756e26 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.182284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334aea-44d2-4e8b-b552-82bb1b7d0ac4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.212238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.223883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.224777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.225147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.225274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.225581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.226244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.226659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.227065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.227993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=afd5af60bde74ab0a3e8f3f2d17ed8ba) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:27.228461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.230303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.243514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.243514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.244188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 262/532] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/role_assignments?user.id=afd5af60bde74ab0a3e8f3f2d17ed8ba&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:27.245907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:27.259488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:27.260675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:27.261703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:27.261703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:27.261703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:27.263201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 271/533] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:27.263201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.354148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.449889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e39229-259a-4899-a0ca-3900eadfd9b9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.466007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6a764-11c9-4732-a1db-7dd4eff48f37 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.495434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.496514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.497011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.497488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.498438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:27.498936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.500770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.517043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 263/534] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:27.524330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.524894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.527886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.527886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.527886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.527886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:27.527886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.528446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.544965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 272/535] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:27.556417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.618601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.699459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72345d37-2980-4409-a881-4e95fb22d7b6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.741155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.742374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.742887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.743426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.751948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.754016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:27.768521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:27.769501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:27.770326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:27.773025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 273/536] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:27.775923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6bca8-3a23-4c3f-8e8b-59d58390fbf5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.778660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.850074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.875713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.876227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.876504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.876865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.896656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.898294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.899711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:27.904847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92366106-d0b0-4e34-827e-387156cdf946 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:27.918591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:27.919559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:27.920377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:27.932439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 264/537] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:27.937969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:27.950936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:27.952042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:27.952538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:27.952999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:27.953911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:27.954419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:27.956247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:27.971058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 274/538] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:27.980931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:28.131878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d3038-e0f1-46cf-b032-7cea5e00abf9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:28.179606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:28.180799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:28.181118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:28.181456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:28.182318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=e9c49b9be98a4a9194239c5be186e1ac) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:28.182632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:28.185938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e935d-d58a-4d5f-941e-0ee5999cab39 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:28.188607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:28.202255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 275/539] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/role_assignments?user.id=e9c49b9be98a4a9194239c5be186e1ac&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:28.262898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:28.263560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:28.263837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:28.264104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:28.264894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:28.265198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:28.266935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9291b3d-a9d9-4805-98f9-25e71f56560b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:28.300156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 265/540] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:28.306435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:28.326213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add288b-5baf-41c9-8c04-657ece665af3 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:28.326421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add288b-5baf-41c9-8c04-657ece665af3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:28.326722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add288b-5baf-41c9-8c04-657ece665af3 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:28.328866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 266/541] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:39:28.339889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:28.339889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:28.339889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:28.361299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:28.364407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:28.365680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:28.447894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:28.519310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 267/542] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:28.532076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887b93a-7a48-4eee-94a0-8308ddf49306 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:28.589284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:28.589912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:28.590191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:28.590462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:28.591210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=65d52c53e1244d25b0e16374101ea139) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:28.591558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:28.593092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:28.604472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 276/543] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:28 2022] GET /identity/v3/role_assignments?user.id=65d52c53e1244d25b0e16374101ea139&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:28.921220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:29.036210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4e9e-d01d-4713-aef0-8cbbce8ab151 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:29.082825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:29.083589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.083871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.084141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.097231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:29.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:29.103416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:29.104201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:29.109094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 268/544] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:29.110083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:29.225778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:29.252158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aed4e-f574-4d9d-8290-b8e4c0141a4e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:29.318701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:29.319349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.319626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.319954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.320637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:29.320941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:29.322350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:29.336047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 277/545] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:29.342129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:29.424890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:29.551108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8153c3-fd60-4761-a9da-85ce754b4906 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:29.600038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:29.601165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.601640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.602104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.619161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:29.621089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-0588c091-39d1-45c0-b695-32b0f83b8e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:29.638281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:29.639248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:29.640168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:29.640168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:29.640168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:29.642358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 269/546] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:29.648630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:29.782838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0c42e-a82c-4249-9832-859e7e95ff0d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:29.831894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:29.832902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.833400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.833929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.834787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:29.835368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:29.837012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:29.855393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92390cb-7212-4d4a-abb1-d7a774732354 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.856039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92390cb-7212-4d4a-abb1-d7a774732354 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.856039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92390cb-7212-4d4a-abb1-d7a774732354 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.857599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 270/547] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:39:29.863907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:29.864195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:29.864453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:29.869278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 278/548] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:29.874984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:29.886401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:29.888575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:29.891597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:30.045027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c85cb-4b28-45f9-8e3f-169e8f3f8421 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:30.089281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:30.090270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.090553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.090825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.093731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 271/549] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:30.102121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:30.103654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:30.117082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:30.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:30.119382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 279/550] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:30.126264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:30.169320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:30.387072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6541c-f71c-49b3-925f-50eb967b874f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:30.407421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:30.491930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:30.493030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.493471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.493903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.495312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:30.495821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:30.498620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:30.570235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 272/551] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:39:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:30.573279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:30.576625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:30.629044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9945f2-f88f-42e1-b149-581cd361f1c4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:30.678917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:30.680302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.680783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.681713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.695580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4b943efd-7d90-4f34-a7a9-3e9e05b8c3a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:30.702912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:30.703939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:30.704758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:30.713109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 280/552] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:30.714187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8822c-42a6-474c-9245-e025453d217f None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.714658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8822c-42a6-474c-9245-e025453d217f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.715116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8822c-42a6-474c-9245-e025453d217f None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.717985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 281/553] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:30.721434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8822c-42a6-474c-9245-e025453d217f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:30.737467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:30.786995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed8c25-d8fd-4c64-bb45-e8327c79424d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:30.831931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:30.832559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.832885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.833160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.899485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8822c-42a6-474c-9245-e025453d217f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:30.919614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:30.921086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e751d-f506-45f9-9ce0-56d0d4f72ff7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:30.949479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:30.964008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:30.965047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.965527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.966009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:30.967240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:30.967780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:30.970431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8775ef-f3fa-455f-a573-a6b8142ab6a2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:30.989097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 282/554] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:30.989097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:30.989097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:30.989097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.014935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:31.016199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:31.024795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.062269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 273/555] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:31.063385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ea73-bd51-4119-b829-f512c6bc93fd None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.063667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ea73-bd51-4119-b829-f512c6bc93fd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.063933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ea73-bd51-4119-b829-f512c6bc93fd None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.065507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 274/556] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:31.066419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54ea73-bd51-4119-b829-f512c6bc93fd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.135916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 283/557] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:31.137294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.217071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54ea73-bd51-4119-b829-f512c6bc93fd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.263613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.264404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.264744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.265079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.278329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:31.280230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-52e8f62d-a87e-4bf7-8fdc-aacbea9cf47b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:31.300043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:31.301714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 275/558] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.303747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.324839 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65bd72-7d77-4c7d-af22-9a06d5acdd71 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.328640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:31.329896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:31.356140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.367266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.393243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.395576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.396478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.397224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.417421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:31.420899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:31.423644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 276/559] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:31.424626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:31.450887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:31.452875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:31.454508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:31.454508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:31.454508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:31.458033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 284/560] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:31.467680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.552115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a9ca-b28f-4027-a3ca-6ddf24e377eb None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.599291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.599291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.599291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.599291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.599910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:31.599910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:31.600402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:31.615040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 277/561] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:31.620381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.640249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.688710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cfcca-ba39-4e1a-a1ac-32fba1609c86 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.742256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.743305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.743783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.744294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.744953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:31.745357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:31.746672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2602cc-0b0f-4a3a-bee1-6ac228fc4e2d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:31.751412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df931-22c9-41db-8f26-49ff8dbbccf5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.769824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 285/562] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:31.771178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861091d-554f-4926-98ba-c282c968d866 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.771462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861091d-554f-4926-98ba-c282c968d866 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.771725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861091d-554f-4926-98ba-c282c968d866 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.773291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 286/563] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:31.775437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861091d-554f-4926-98ba-c282c968d866 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.811760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.812440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.812723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.812994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.813805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.824285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:31.825825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:31.839276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:31.840926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:31.843178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:31.845508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 278/564] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:31.846571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:31.941548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861091d-554f-4926-98ba-c282c968d866 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:31.972911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:31.978507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:31.979154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.979471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.979750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:31.990460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5219c038-d4ac-4d36-9ada-935433ce7ba7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:31.995483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:31.996400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:31.997218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 287/565] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:31.997940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:31.998009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:31.998309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.016531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f9866-deba-42d7-aa23-52f45928af6c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.017844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:32.019088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:32.059261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.059885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.060157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.060423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.068743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.070659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:32.083086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:32.085364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 279/566] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:32.086804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.115731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 288/567] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:32.115731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.156089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.209951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579694-9f1b-414d-8c17-e4cf8df931f2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.255930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.257680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.258117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.258545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.259337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:32.259649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.261147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.263851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.281776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6e601-dad0-402d-87ab-f2f7f17f2f49 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.286539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 280/568] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:32.287391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.352386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.353050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.353326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.353599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.366357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:32.371583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:32.373166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:32.374201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:32.377840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 289/569] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:32.378824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.434276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53906a-0fa4-400c-9e99-e628a9a2c2e1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.474538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.475164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.475391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.475530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.475735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.476459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:32.476830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.478758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.487606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 281/570] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:32.488491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.499578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749084b2-bd3d-40e3-ad7e-65a7cf46e85d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.537761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.538337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.538574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.538838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.539580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:32.539913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.541415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.556359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 290/571] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:32.557973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.562376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.617106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59015dd4-f1b4-4952-b5ef-107f2b4fa9f1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.656900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.657892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.658330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.658764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.672656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.673775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd089969-bd2e-4212-839e-d0262735fd94 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.674128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:32.690888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:32.692160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:32.693645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:32.693645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:32.693645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:32.693645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 282/572] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:32.693645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.712022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.712640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.712922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.713190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.713862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:32.714164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.715604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.729411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 291/573] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:32.729411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.733872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.809202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a326bc-0a5a-45cf-8406-e963d3913f2f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.816743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:32.844244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d075e70-fd63-4a4c-a4b3-6310f86a8e82 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.846119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.846767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.847041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.847339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.858619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:32.864077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:32.865087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:32.865917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 283/574] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:32.866684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.887417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:32.888072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:32.888349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:32.888649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:32.900386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:32.901878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:32.918614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:32.919932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:32.921299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:32.921299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:32.921299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:32.921299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 292/575] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:32.921299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:32.978269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c31646-69db-431b-ac18-5b29f0bfe9b0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:32.982250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.019206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.019818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.020098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.020379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.021107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=3cb125ada8d64bc49ff99383d05aa69e) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.021411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.022962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.030110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 284/576] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/role_assignments?user.id=3cb125ada8d64bc49ff99383d05aa69e&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.030927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ddde-99a3-4262-b6c2-aa4235d2334a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.031572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.086270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.087425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.087927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.088422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.089681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.089681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.091260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.101701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.114888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 293/577] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.114888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.213856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3676fb-52f5-41ab-98a9-830ff24cea00 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.256347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141af6ca-7c16-4cfc-9e74-7149b14f2254 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.256981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.257596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.257798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.258031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.270340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.271863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.284548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:33.291849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:33.293840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:33.296021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:33.296021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:33.296021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:33.296021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 285/578] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:33.296021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.315279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.315926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.316199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.316469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.317164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.317465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.318916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.333791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 294/579] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.333791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.439566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.450746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930bb2-c33a-42e9-934d-c977e11fa862 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.465111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e25a54-1d0b-4234-83f9-130917283f8f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.497099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.497750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.498024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.498282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.506397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.508337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.515089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.516150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.516639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.517122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.518310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.518829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:33.520984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:33.522122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:33.524076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:33.524862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.528353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 295/580] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:33.529936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.551412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 286/581] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.551412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.578865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.666990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2123c-dfc6-42e5-89a5-c26ca0beb3e3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.692987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa342a1-f51f-4d91-8ba1-84a6dbfd6a11 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.714388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.715288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.715730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.716127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.717408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.717726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.719372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.735309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 296/582] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.737725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.742108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.742879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.743157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.743570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.756759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.758385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.760932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:33.776922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:33.778323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:33.779705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:33.779705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:33.779705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:33.779705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 287/583] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:33.779705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.904228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:33.909181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a318b-8a17-4c7c-beae-fa2bfd90b581 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.919629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849e00-20b4-40a1-ac1c-c9511da71458 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:33.950125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.950921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.951189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.951859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.952580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:33.952878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.954366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.959603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:33.960334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:33.960638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:33.960909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:33.973201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 297/584] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:33.973201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:33.975956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:33.977987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:33.994602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:33.995694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 288/585] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:33.997230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.078661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.106836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab22ad0-94e9-4a84-9792-9d063b8bf9c3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.147082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.147814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.148086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.148353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.155434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01a088-fd13-4ae1-b7b4-5440bd22e2b1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.156995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.158696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:34.174173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:34.176029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:34.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:34.180150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 298/586] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:34.181891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.213793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.214879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.215535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.215814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.216071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.216718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:34.217013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.218416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.241173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 289/587] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:34.241173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.341833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89bef-f1e7-4773-82e8-242df0aa8a95 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.378946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ee070-0b5a-4e7a-bd4b-55293882f20b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.400130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.400969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.402412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.402803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.403021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.426416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=97e8cd3ae37443acbb2c7c710646686a, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.430002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.434219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.434991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.435436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.435802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.436657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:34.437049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.438713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.467285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 299/588] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:39:34 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:34.467285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.467719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 290/589] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:34.467719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.615464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.644390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-2ac1-46f1-8521-51ac12c210fa None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.680042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd256a7-7302-4625-ba19-a8dfebe27ffd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.703739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.704512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.704857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.705138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.715272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.716913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:34.733138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:34.734147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:34.735078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:34.738529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 291/590] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:34.739552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.745395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.745395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.745395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.745910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.746386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:34.746695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.748444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.782251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 300/591] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:34.785262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.787166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.871631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc653e9-1266-4785-8cc9-416318311021 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.923583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.924250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.924532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.925036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.930608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f705656-9c0b-4515-bbe7-a380f33d2919 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:34.939973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.942125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:34.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:34.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:34.963246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:34.963246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:34.963246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:34.966691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 292/592] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:34.967889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:34.974301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:34.986406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:34.987573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:34.987998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:34.988410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:34.990003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:34.990754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:34.992710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.016123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 301/593] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:35.017883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.086133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:35.108715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d5fb-d205-434c-a6a8-a84af4cc7f9e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.156276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.156962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.157409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.157812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.167049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923a0ca-8f64-4bcb-86b4-c1a93c4f2a65 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.170905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.173579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:35.193448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:35.194412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:35.195421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:35.195421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:35.195421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:35.195421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 293/594] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:35.195928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.245115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.246481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.247167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.247902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.249264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:35.249845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.251763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.276342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 302/595] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:35.278110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.335613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f2ae5-a857-476c-bbda-637ffed22b58 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.375793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:35.376452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.376656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.377093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.377337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.385291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:35.404244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.406785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:35.431731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:35.432747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:35.433612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:35.433612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:35.433612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:35.433612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 294/596] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:35.433612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.496336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71bd56-3944-47e2-abc6-78ffae5c2f8d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.561062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.562249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.562730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.563213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.564421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:35.564961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.567386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c00a4-8e2a-4c7f-b0fb-b9d480411d12 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.582500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638d895-95ee-4361-83cc-c84121ecf657 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.596418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 303/597] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:35.596418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8246-ff62-49ec-a066-a467911b79ea None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.596418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8246-ff62-49ec-a066-a467911b79ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.596418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8246-ff62-49ec-a066-a467911b79ea None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.599493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 304/598] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:35.602093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8246-ff62-49ec-a066-a467911b79ea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.633363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.634182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.634460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.634778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.643530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.645016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:35.659536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:35.661474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 295/599] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:35.664009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.765891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:35.774498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8246-ff62-49ec-a066-a467911b79ea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.778938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:35.817473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.818091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.818396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.818638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.819311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:35.819698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.821220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf7a92-ba2d-4be9-9aaf-f228d85f4a3e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.841902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 305/600] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:35.841902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.841902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.841902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.857086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:35.858328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:35.888357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442058-1c14-44f7-a9b4-94334b449ae9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:35.939699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:35.940420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:35.940696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:35.940965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:35.948863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:35.950387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:35.963532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 306/601] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:35.963532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:35.975602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:35.977537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:35.979415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 296/602] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:35.980305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.172974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:36.189210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fd1c4-e4b3-4165-b284-723106b4bb45 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.216070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ee91a-2d74-44c2-be20-9c1490008066 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.228774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:36.264800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.265426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.265693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.265956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.266658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:36.266954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.268439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.268600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.270041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.271128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.272693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.287290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 307/603] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:36.287904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.288285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.291856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:36.309453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:36.311245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:36.312844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:36.315526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 297/604] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:36.316470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.440199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec30eb9-c556-4ce1-b061-4b4b19f19c17 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.485156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.486391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.487373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.487665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.488361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:36.488661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.490285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.498404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1d35-e5a3-4cc8-aa71-0c64564fb4d2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.505783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 308/605] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:36.506582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.527807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:36.571810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.572881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.573371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.573847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/afd5af60bde74ab0a3e8f3f2d17ed8ba/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.598782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=afd5af60bde74ab0a3e8f3f2d17ed8ba, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.601865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.626798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 298/606] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:39:36 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/afd5af60bde74ab0a3e8f3f2d17ed8ba/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:36.628356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.675589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:36.681010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acff19-b59f-4ae9-a859-14913bbc7785 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.767395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.769751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.770945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.771639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.773598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:36.774412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:36.778371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:36.812041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 309/607] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:36.814276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.865916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7849e-a630-45b3-910a-7f555ade5aa4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:36.924248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:36.925172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:36.925562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:36.925955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:36.955156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:36.963967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:36.964977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:36.965870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:36.966506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 299/608] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:36.972278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:36.972592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:37.021739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4c77-9ff9-4f80-966a-dc6961c3b615 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:37.092238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:37.092924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.093195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.093466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/e9c49b9be98a4a9194239c5be186e1ac/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.108723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:37.113872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=e9c49b9be98a4a9194239c5be186e1ac, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:37.115639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0017350-3797-4e70-bcef-017acdccb18f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:37.140656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 310/609] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:39:36 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/e9c49b9be98a4a9194239c5be186e1ac/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:37.140656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd604-adde-4c5e-8b85-8b4d6339d7f2 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.140656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd604-adde-4c5e-8b85-8b4d6339d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.140656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd604-adde-4c5e-8b85-8b4d6339d7f2 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.143417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 311/610] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:37.144333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fd604-adde-4c5e-8b85-8b4d6339d7f2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:37.160181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6541f8-3d5a-45c1-b510-4a79518d0189 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:37.236646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:37.237908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.238416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.238899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/65d52c53e1244d25b0e16374101ea139/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.270371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=65d52c53e1244d25b0e16374101ea139, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:37.273791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30b730-a778-4aae-842b-630613d584ca None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:37.300811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 300/611] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:39:36 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/65d52c53e1244d25b0e16374101ea139/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:37.303691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.304278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.304719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.360854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:37.362559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:37.411277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fd604-adde-4c5e-8b85-8b4d6339d7f2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:37.496840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:37.531337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:37.532335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.532822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.533315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.534844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:37.535270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:37.538007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:37.557894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 312/612] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:37.568089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:37.574177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 301/613] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:37.818588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85655d-56e0-4338-8081-04b1c70f00a5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:37.871562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:37.872224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:37.872498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:37.872764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:37.895777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:37.897295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:37.918204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:37.920022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:37.921746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:37.921746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:37.921746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:37.921746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 313/614] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:37.931753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:38.030147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:38.101690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4b29e-b877-47af-92ee-2f78b8700cdc None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:38.166313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:38.166950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:38.167255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:38.167517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:38.168184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:38.168483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:38.169883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:38.184040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 302/615] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:38.192891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:38.196410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:38.283822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:38.365145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f8107-d6c3-4c78-a152-d90dbd17539d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:38.463860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:38.464942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:38.465428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:38.465897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:38.508345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:38.519623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba23dbf-2f52-499b-816e-1ba2ab54747d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:38.520368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:38.521952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:38.523702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 314/616] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:38.534419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:38.559966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:38.596594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:38.597700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:38.597700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:38.598189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:38.613361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:38.615926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:38.651249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:38.652956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:38.654670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:38.663590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 303/617] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:38.669450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:38.839534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:38.843963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269275-cfa6-458d-be80-08c5ffeb9bf9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:38.920399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:38.920922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:38.921193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:38.921464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:38.922141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:38.922428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:38.923885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:38.933622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76377582-80db-4fad-85c7-48cf2a73c1a3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:38.945811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 315/618] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:38.954174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.011104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.015320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.024479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.025489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:39.028551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:39.029553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:39.033948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:39.074559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 304/619] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:39:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:39.085529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.155515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:39.218448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4739e-ae7e-4d18-97a0-8307ad43de6f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:39.346350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cd25e-d0de-48c0-81cd-94c73b65828f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:39.350584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.353747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.354766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.355741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:39.382791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:39.386249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:39.416815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:39.418520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:39.420167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:39.420167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:39.420167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:39.420167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 316/620] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:39.429642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.431416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.432542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.433014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.433497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:39.468234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:39.481722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:39.484349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:39.523563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:39.525388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:39.527129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:39.527129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:39.527129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:39.531613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 305/621] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:39.541336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.695543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:39.714498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b47302-1157-4562-b0db-40320fafdff9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:39.759486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.760545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.761326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.761860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:39.762870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:39.763575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:39.765812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:39.781208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 306/622] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:39.781403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8f870-eaf0-43a1-8bb6-91a02b7e2c8a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:39.787837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.873290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.874292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.874698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.875099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:39.876159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:39.876609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:39.886032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:39.918085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 317/623] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:39.926635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:39.938597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa6b2-ac9c-46b4-8396-a1614a92aa2e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:39.988351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:39.989028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:39.989299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:39.989565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.005736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:40.009985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:40.036343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:40.037318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:40.038230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:40.039127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 307/624] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:40.045817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:40.078632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:40.097629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:40.142323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1444c-b25c-4147-b513-498c265767ab None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:40.192569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:40.194109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.194668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.195384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.205547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:40.208040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:40.216507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a6543-980c-4de0-b202-e279d45d65ae None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d994178e-4550-4885-b8b1-f11f1d2a14c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:40.226518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:40.227695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:40.228632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 318/625] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:39:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:40.230248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0569-6fee-4b35-a7d8-b5f064bc3df8 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.230248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0569-6fee-4b35-a7d8-b5f064bc3df8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.230490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d0569-6fee-4b35-a7d8-b5f064bc3df8 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.232003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 319/626] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:40.232893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087489-725b-467c-bcf4-c9a3f5ed0e77 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.233167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087489-725b-467c-bcf4-c9a3f5ed0e77 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.233433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087489-725b-467c-bcf4-c9a3f5ed0e77 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.234811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 320/627] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:40.242364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023395-543e-4d35-b550-913715ecda86 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.242856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023395-543e-4d35-b550-913715ecda86 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.243404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17023395-543e-4d35-b550-913715ecda86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.268359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:40.269264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.269753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.270188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.271505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:40.272031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:40.274260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-17023395-543e-4d35-b550-913715ecda86 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:40.274711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:40.276324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-17023395-543e-4d35-b550-913715ecda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:40.303785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 308/628] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:40.303785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:40.403716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:40.413727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 321/629] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:40.416595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b09e-85ad-403f-a709-b1df1ef6396b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.417412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b09e-85ad-403f-a709-b1df1ef6396b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.418085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b09e-85ad-403f-a709-b1df1ef6396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.452525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4d94b09e-85ad-403f-a709-b1df1ef6396b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:40.455081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a63a1-be15-4b06-b67f-214b7ef5b33f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:40.455521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4d94b09e-85ad-403f-a709-b1df1ef6396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:40.494499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:40.495405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.495779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.496146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.497014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:40.497406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:40.499096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:40.525353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 309/630] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:39:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:40.526363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:40.569942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:40.591277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 322/631] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:40.591277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0947-1919-44b4-8345-e9f755c4b183 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.591277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0947-1919-44b4-8345-e9f755c4b183 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.591277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f0947-1919-44b4-8345-e9f755c4b183 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.597822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 323/632] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:40.597822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f0947-1919-44b4-8345-e9f755c4b183 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:40.654166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca8ae-3e0f-4018-9da5-644c46124cc3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:40.691999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:40.704321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:40.705761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.706239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.706686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/3cb125ada8d64bc49ff99383d05aa69e/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.725440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=3cb125ada8d64bc49ff99383d05aa69e, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:40.727299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1ee64-dda7-457c-bcc6-6522785e5a4b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:40.746799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 310/633] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:39:40 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/3cb125ada8d64bc49ff99383d05aa69e/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:39:40.748836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.749882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.750377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:40.773836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:40.775587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:40.827690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f0947-1919-44b4-8345-e9f755c4b183 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:40.875445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 311/634] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:40.894931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:40.898709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:40.900201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:40.900534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:40.900903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.018549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.033452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:41.034867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:41.096089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955572d-05ee-486b-b18f-01c2bb783319 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.105621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.143655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.144318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.145192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.145478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.160471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:41.166972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:41.168989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 312/635] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:41.169847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.176125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 324/636] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:41.177127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.301031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b946-fab8-40cb-861e-3743c6d6f2fd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.303715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.315752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564134-5835-474f-8a8f-1f7be460fb10 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.351829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.352477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.352752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.353022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.365857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:41.367890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.368701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.369147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.369465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.370339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:41.370864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:41.371031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:41.375112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:41.379310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 313/637] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:41.380629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:41.380753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.393643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 325/638] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:41.393643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.416833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.528319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a2a52-448b-4a66-8a31-38486ed00385 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.529993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af98a6-9a93-4e8f-908b-9a5d01fc8c67 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.570733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.571438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.571718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.571980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.583980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:41.585536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:41.589770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.590843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.591368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.591667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:41.604995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:41.607077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:41.609130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:41.609130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:41.609130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:41.609130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 314/639] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:41.609130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.615336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:41.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:41.627432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:41.629629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 326/640] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:41.637308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.656216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.720842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:41.754416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9eab0a-f52c-4c06-8671-932b39b1450b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.767380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a405-7040-48dc-bee7-353307834335 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:41.811649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.812304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.812613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.812878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.823651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:41.824628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:41.825108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:41.825578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:41.826205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:41.826869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:41.827486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:41.827706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:41.830832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:41.842786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:41.851630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:41.853390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:41.853390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:41.853390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:41.853390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 327/641] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:41.853390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.859001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 315/642] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:41.865219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:41.949130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.005295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85d1ea-0397-4e64-a87d-e6ccb64fc82c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.018652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27518657-dac4-4784-ae6e-fe18befa23a6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.059672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.060768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.061235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.061906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.062530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:42.062926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.067468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.071497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.072442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.072862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.073263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.074141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:42.074531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.076885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.082151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 328/643] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:42.094765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 316/644] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:42.094999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.095151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.095151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.244717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190a6c6-8a2d-48ad-bcae-254cf31627cc None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.248015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca023-e95a-455b-b6e4-0e8e1771ce1d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.279192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.289875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.290497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.290763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.291026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.291723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:42.291997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.293193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.293393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.294138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.294558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.295300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.306164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 329/645] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:42.307011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.315860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.317343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:42.336382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:42.337851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:42.339092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:42.339092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:42.339092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:42.339092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 317/646] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:42.339092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.416236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.461047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a58ae5-7c72-4654-a330-a8cfbcdbd1e0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.473521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30342d-93ad-4504-88a7-c42fe1550655 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.520163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.520919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.521257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.521592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.522707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.523484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.523828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.524157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.530020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.531742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.538760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.540536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:42.546119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:42.548153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:42.550151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:42.555890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 330/647] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:42.557026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:42.565311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:42.566851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:42.568740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:42.568740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:42.568740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:42.568740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 318/648] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:42.568740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.619099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.681811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f0456-fa0f-49c9-95fb-1d9c2fb84a9d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.703827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020b2b0-6943-427c-8acc-f446f9cf789c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.716268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.724839 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.725630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.725827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.726098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.735094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.736625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:42.751212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:42.752340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 331/649] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:42.754068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.757752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.758399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.758679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.758947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.759628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:42.759931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.761340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.783094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 319/650] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:42.783094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.878120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016b725-7476-4ffa-98e0-ca8ecc92b1a0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:42.886317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:42.918909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:42.919552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:42.919857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:42.920102 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:42.920776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:42.921077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:42.922521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:42.936298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 332/651] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:42.937160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:42.957754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017136-27f4-4a69-901b-40b21ce6b697 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.032067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.032848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.033207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.033573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.034391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:43.034784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.039386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.067324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 320/652] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:43.067324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.090367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a5c11-2942-4401-b7b3-9c81dd1cab14 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.104010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.148791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.149629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.150057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.150539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.152466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:43.152954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.155521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.184560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 333/653] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:43.185934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.225634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.227955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be9935-975d-46b4-975b-3a0afa0b299b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.273092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.274083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.274454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.274894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.287749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.290055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:43.307953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:43.314905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 321/654] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:43.316081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.334418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b133-fe90-4cde-a1d8-4a6cc39a0ed6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.370967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.371614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.371886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.372169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.372911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:43.373218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.374721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2abb04-845c-49d2-acb6-e6a4de715b53 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.382139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 334/655] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:43.384265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704e1e-c6fb-43b2-bc2c-81aae9fe8c20 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.384634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704e1e-c6fb-43b2-bc2c-81aae9fe8c20 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.385083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704e1e-c6fb-43b2-bc2c-81aae9fe8c20 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.386774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 335/656] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:43.387901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704e1e-c6fb-43b2-bc2c-81aae9fe8c20 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.423939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.447340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983176e-90a3-4d46-8425-8d36808246b3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.512519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.513711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.513889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.514276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.514654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.525733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.527242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.535798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704e1e-c6fb-43b2-bc2c-81aae9fe8c20 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:43.541471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:43.549598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 322/657] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:43.550762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.601039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.601661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.601943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.602177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.710817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.713543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.716880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c79a15-6050-40cb-9673-e5f5feacb926 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:43.773414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:43.774440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.774440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.774682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.775325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:43.775631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:43.777085 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4493c-c4da-4cb8-81a3-f1d61d659545 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:43.799659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 323/658] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:43.801124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:43.801538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:43.801941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:43.813606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.832279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:43.834184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:43.863020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:43.884727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 336/659] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:43.885747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:43.959303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 324/660] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:43.959303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.073378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac262bd-268e-42e6-a765-8414ffba28b5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.107158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781915bb-36a1-4bf0-bc1e-33d9603d1baf None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.114810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.115716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.116184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.116618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.117503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:44.118075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.119717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.144330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 337/661] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:44.146282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.150047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.150860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.151263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.151697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.154764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.231512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.235901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.237763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.279308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8e0af-0fad-41fc-9b63-a8f4c55eac0e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.324533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.325346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.325725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.326140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.338984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.340668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:44.356148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:44.357691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:44.359415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:44.359415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:44.359415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:44.363588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 338/662] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:44.364568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.376318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 325/663] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:44.379064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.442891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.508448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.514469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4dd33-b647-4458-be68-30dac88db8ae None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.545354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1bfa-a134-455f-897e-d8d8c1afe794 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.565950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.566690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.566969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.567240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.591108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925884-8199-493e-8468-5c674432c32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.592375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925884-8199-493e-8468-5c674432c32b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.592856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925884-8199-493e-8468-5c674432c32b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.593275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925884-8199-493e-8468-5c674432c32b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.606335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5925884-8199-493e-8468-5c674432c32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-e5925884-8199-493e-8468-5c674432c32b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:44.611730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:44.612926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:44.613953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:44.621540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 326/664] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:44.621540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925884-8199-493e-8468-5c674432c32b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.666535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.668235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.736406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.747976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925884-8199-493e-8468-5c674432c32b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.790664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.791362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.791579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.791844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.792499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:44.792803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.794198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.807327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 327/665] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:44.812147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.818307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:44.824573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 339/666] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:44.825581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:44.944958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374f65-3ee2-4512-b1dd-d06172882077 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.948419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f97f6-41fc-40f8-b081-a958902d8f01 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:44.985433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.986081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.986363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.986638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.991440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:44.992193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:44.992620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:44.993016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:44.993831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:44.994260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:44.996128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:44.999190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.000652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.011080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 340/667] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:45.011080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:45.021302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:45.027535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:45.028680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:45.028680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.028680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:45.028680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 328/668] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:45.028680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.040647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.106026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.152598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62add39-83c6-4af1-b957-c1b9e497064e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.171766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6ca9-a3a1-4342-8bfb-edc6c61520f1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.192170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.192884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.193195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.193452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.201586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.203075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:45.215927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:45.217437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 341/669] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:45.218442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.222422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.223509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.223848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.224134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.241166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.242911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:45.261603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:45.262622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:45.263519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:45.263519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.263519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:45.263519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 329/670] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:45.263767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.335115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.354955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203ebb-3c3e-415d-92d5-c7cb40119982 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.393361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.394335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c331c33-ac8e-4cf0-a964-a45430791c1a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.421478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.422566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.423044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.423557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.448689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.449334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.449679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.449949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.450646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:45.450971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.452620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.455825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.458215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.475977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 330/671] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:45.475977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:45.489494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:45.491514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:45.493750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:45.493750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:45.493750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:45.495836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 342/672] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:45.498543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.616765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdb89c-13cb-4975-9623-3eb69c1c1482 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.644731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.667645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc8e76-13d8-4a59-9afb-965690cf59f8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.670323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.671025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.671394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.671735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.672446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:45.672742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.674206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.696680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 331/673] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:45.696680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.713894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.714439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.714717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.714993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.715703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:45.716010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.717526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.730861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 343/674] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:45.731782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.787668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:45.839246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ada97-4668-499e-829b-c74c071070b4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.874129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.874778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.875055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.875347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.875749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4e877-92ac-4833-a839-dae0d6177c19 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:45.882869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.884373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:45.899156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:45.900187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 332/675] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:45.908431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.917179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:45.918070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:45.918432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:45.918830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:45.919640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:45.920078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:45.921934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:45.935550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 344/676] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:45.936482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:45.948772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.038619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97121209-6d47-4260-9ce3-633f623128e0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.083523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.094405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8296-084b-4e9d-9b69-a97a6b97d86f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.100486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.101132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.101407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.101674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.102445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=afd5af60bde74ab0a3e8f3f2d17ed8ba) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.102757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.104338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ea7e8-43e7-4ddc-a840-e16952ef7db7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.112471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 333/677] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/role_assignments?user.id=afd5af60bde74ab0a3e8f3f2d17ed8ba&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.114027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda11d5-c1df-4180-acb4-46a7805d8a57 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.114245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda11d5-c1df-4180-acb4-46a7805d8a57 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.114547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda11d5-c1df-4180-acb4-46a7805d8a57 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.116011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 334/678] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:46.116795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda11d5-c1df-4180-acb4-46a7805d8a57 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.162412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.163297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.163648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.163992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.173109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.174833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:46.189045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:46.190067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 345/679] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:46.191692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.294901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda11d5-c1df-4180-acb4-46a7805d8a57 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.297134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.341223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc1ad9-29a1-43e6-ac38-25b6072aba5c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.348218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.349157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.349600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.350098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.351418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.351862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.353929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032c28-2734-4b00-954d-cdcea8eddad2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.376858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 335/680] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.378669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.379119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.379609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.387599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.388388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.388719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.389047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.399708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.401209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.403073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:46.404747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:46.421392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:46.422467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:46.423430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 346/681] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:46.424913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.451292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.531406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 336/682] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:46.531406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.582472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67eee5-d439-4d60-acfc-b6aa40a8ec4b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.626447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.627168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.627542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.627876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.628816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.628989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.630542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.650191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 347/683] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.651610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.728896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.737787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:46.748237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f11262-e137-41e0-8358-181a69ea8361 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.773573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00038-4e8f-43ba-8c9d-015667a22ee1 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:46.813047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.814273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.814827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.815439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.816138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:46.816926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:46.817040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.817431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:46.817778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.817960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:46.818839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=e9c49b9be98a4a9194239c5be186e1ac) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:46.819370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:46.821015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.821301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:46.828665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 337/684] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/role_assignments?user.id=e9c49b9be98a4a9194239c5be186e1ac&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.829864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:46.864636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 348/685] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:46.871303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:47.048079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffda3-968f-42d8-9657-6a594979d3af None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.099592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.100364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.100647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.100917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.101690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=65d52c53e1244d25b0e16374101ea139) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:47.101996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:47.104488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:47.111871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 338/686] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/role_assignments?user.id=65d52c53e1244d25b0e16374101ea139&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:47.113030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:47.158780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587c9b-5551-4115-ba76-771f37eae767 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.164347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:47.200279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:47.208188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.208507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.208751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.209084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.310701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:47.314477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:47.331431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a65b-5a29-4ae9-a069-1ed03f6b4bf1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.401003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.402046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.402498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.402914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.440173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:47.445752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:47.448783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:47.450792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 339/687] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:47.457576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:47.582313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 349/688] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:47.588111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:47.652789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:47.663767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:47.691163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ef95c-f4b8-4b04-a374-1f53a0903581 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.767301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.768660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.769113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.769575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.770904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:47.771611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:47.774316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:47.784705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3d7e8-7eea-40ce-b642-ef746cb16b96 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:47.790203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 340/689] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:47.800945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:47.932476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:47.933558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:47.934040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:47.934506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:47.976895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:47.979529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.006445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8a85a-85be-47dd-94a2-985e28407462 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:48.028855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:48.030704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:48.032522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:48.032522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:48.032522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:48.035263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 350/690] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:48.043320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.109480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.114448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.114898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.115319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.136933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.142988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.145298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:48.181379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:48.182854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:48.184344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:48.184344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:48.184344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:48.199240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 341/691] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:48.205737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.345628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:48.389335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3ae6a-e71b-4703-b4ed-2de580a6689f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.457452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.458488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.459187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.459769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.460999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:48.461511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.464882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.495600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 351/692] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:48.499287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.532542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36571dd-1727-4dc3-9e5e-e8d4781414ea None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.663105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077603-8265-4226-b576-e7beabbac363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.664660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077603-8265-4226-b576-e7beabbac363 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.665757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077603-8265-4226-b576-e7beabbac363 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.667794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25077603-8265-4226-b576-e7beabbac363 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.669455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25077603-8265-4226-b576-e7beabbac363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:48.672396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25077603-8265-4226-b576-e7beabbac363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.675180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25077603-8265-4226-b576-e7beabbac363 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.719477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 342/693] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:48.727666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077603-8265-4226-b576-e7beabbac363 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:48.769245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f91fee-fef0-4ccb-83db-76280581f84d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:48.835159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:48.850441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:48.851877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.852340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.852814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.869081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:48.872455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-6170b4bb-c142-4074-83b6-0ccea07ec7e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:48.895726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:48.897498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:48.899146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:48.903481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 352/694] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:48.906012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4282-f4fe-4902-b234-c8390de45ddd None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.906789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4282-f4fe-4902-b234-c8390de45ddd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.907547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4282-f4fe-4902-b234-c8390de45ddd None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.910522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 353/695] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:48.919416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:48.919416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:48.919416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:48.928473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:48.940316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:48.941568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:48.943904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25077603-8265-4226-b576-e7beabbac363 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.013183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.014557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.015033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.015542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.023525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 354/696] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:49.024948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.030999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.033792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:49.073728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:49.077011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:49.080550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 343/697] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:39:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:39:49.091789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.285049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff76742-7d23-4b3e-a96d-610aa2084f2e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.314763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.350981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e4354-a9dc-4054-8b02-1fcfd9371f64 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.364208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.365235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.365760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.366242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.367581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:49.368117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.370717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.408137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.411890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 344/698] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:49.413942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.415714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.416686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.417165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.417626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.418549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:49.419068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.420845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.436520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 355/699] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:49.437454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.558426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fba9a-5bc8-4a5c-8c15-8bb85218c7f6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.612990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0655ce-f1fb-454e-bde4-5eb0798eb014 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.636659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.639062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.640109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.640841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.658597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.659445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.659840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.660240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.675033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.676834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:49.694765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:49.696886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:49.698723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:49.698723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:49.698723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:49.703259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 345/700] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:49.704982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.736031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.768144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:49.780368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.781844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.910650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a9bd-10c5-4d92-985e-5194f449ce8c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:49.942429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 356/701] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:49.942429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:49.974594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:49.975674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:49.976675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:49.976956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:49.977690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=3cb125ada8d64bc49ff99383d05aa69e) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:49.977991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:49.979496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:49.988150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 346/702] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/role_assignments?user.id=3cb125ada8d64bc49ff99383d05aa69e&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:49.989130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.103262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.136006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c556f0-ba1a-4bdf-848a-4e1bf7a7f26d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.154917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.175923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747063c-b580-4913-93df-6d423c3f0a7a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.224132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.228391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.228761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.228974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.254928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.256527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.269176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.269817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.270090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.270347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.271008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:50.271343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.272893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:50.295102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:50.297510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:50.299348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:50.299348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:50.299348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:50.307289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 357/703] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:50.307708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 347/704] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:50.311115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.325575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.479814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1c900-4114-4004-9c0d-b6f005eb763f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.516940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.517561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.517873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.518155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.518737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.519628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6d651-f060-4ee9-a0eb-eac8a8780a84 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.520323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.532302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.549835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 348/705] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:39:50 2022] POST /identity/v3/users => generated 254 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:50.582210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.582855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.583171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.587257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.587257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:50.587257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.587257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.602185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 358/706] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:50.611750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.747041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4cdd-39cf-431c-9c41-584ed6578f6b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:50.790774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:50.791564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:50.791882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:50.792154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:50.801607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:50.803748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2f7e305a-4b07-4493-91c2-796a63b4a4df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:50.820450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:50.822352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:50.825501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 349/707] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:50.830455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:50.851030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:50.972547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269424d6-246c-40fb-84d5-dbde181f47d4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.014373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078a1eb-2ac1-4236-8214-a88a8c504b32 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.015017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078a1eb-2ac1-4236-8214-a88a8c504b32 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.015525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078a1eb-2ac1-4236-8214-a88a8c504b32 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.018351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 350/708] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:51.024522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52c06-43d3-43b1-a46f-3fa3fa8a051c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.025216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52c06-43d3-43b1-a46f-3fa3fa8a051c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.025716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52c06-43d3-43b1-a46f-3fa3fa8a051c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.049425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.050413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.050820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.051255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.052746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-14b52c06-43d3-43b1-a46f-3fa3fa8a051c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:51.052895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:51.053406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.054783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-14b52c06-43d3-43b1-a46f-3fa3fa8a051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:51.056680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3308cc-7274-49dc-a6be-b82d2b614517 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.073268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 359/709] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:51.080166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1a74e-da09-465a-802b-8160e92adecb None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.080805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1a74e-da09-465a-802b-8160e92adecb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.081324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1a74e-da09-465a-802b-8160e92adecb None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.083463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 360/710] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:51.084951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1a74e-da09-465a-802b-8160e92adecb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.153231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:51.176900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 351/711] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:51.179091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.179648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.180112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.200802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:51.202416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:51.239330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:51.286541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 352/712] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:51.324201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1a74e-da09-465a-802b-8160e92adecb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.407554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.408922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.409457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.410137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.411785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:51.412610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.415617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.426979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 361/713] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:51.612551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.736818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.755321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdec80-0cdb-41ac-8dfa-46b73897096b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:51.776265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:51.798807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:51.799789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:51.800285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:51.800696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:51.801530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:51.803735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:51.818238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:51.819672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:39:51.820629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:51.824367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 353/714] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:51.833774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:51.978137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53ab52-a31b-4a35-9828-61d8a6676223 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:52.020695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:52.022017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:52.023312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:52.023927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:52.025426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:52.026097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:52.028237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f59ab-b406-4699-af6e-b9a5d299d891 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:52.029963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:52.044364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 354/715] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:52.053845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:52.108095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:52.108744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:52.109005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:52.109319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] PATH_INFO: `/v3/services/network` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:52.109911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:52.111333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:52.126451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:52.146484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:52.149144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:52.150854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 362/716] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:51 2022] GET /identity/v3/services/network => generated 88 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:52.159262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:52.159681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:52.245200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9ec9-665e-47bf-b4cf-5ca19b35ca49 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:52.323992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:52.324626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:52.324903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:52.325169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:52.325817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:52.326110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:52.329167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:52.345709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 355/717] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:39:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:52.376674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb4091-4605-416d-9776-cf57546d3495 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:52.425868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:52.426506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:52.426778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:52.427436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:52.428213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] RBAC: Adding query filter params (name=network) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:52.428528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:52.430108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c4f1b-e356-4e52-bcc3-0d913ac29301 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:52.435807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 363/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:39:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:52.444533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:52.568021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94e10-cbfe-459c-94d9-2d5fc0ed8d51 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:52.580289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:52.614214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:52.614214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:52.614214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:52.614214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:52.615665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] RBAC: Adding query filter params (type=network) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:52.615665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:52.615665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da688e7-42e2-4c62-a542-2cf4e58f9e4d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:52.621999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 356/719] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:39:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:53.670211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f971a6-9975-45e9-b63e-231cc3527160 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:53.671786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f971a6-9975-45e9-b63e-231cc3527160 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:53.672554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f971a6-9975-45e9-b63e-231cc3527160 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:53.675643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 364/720] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:53.683030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337ab7-3d82-497b-ac99-24fd36375437 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:53.684470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337ab7-3d82-497b-ac99-24fd36375437 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:53.685141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66337ab7-3d82-497b-ac99-24fd36375437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:53.721567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-66337ab7-3d82-497b-ac99-24fd36375437 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:53.723939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-66337ab7-3d82-497b-ac99-24fd36375437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:53.851582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 357/721] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:54.236858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f971a6-9975-45e9-b63e-231cc3527160 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.299886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66ac42-af17-4a6e-bf52-0d851c09e439 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.300566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66ac42-af17-4a6e-bf52-0d851c09e439 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.301024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66ac42-af17-4a6e-bf52-0d851c09e439 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.303357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 358/722] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:54.309341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.309914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.310401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.339344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:54.341595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:54.346219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:54.418200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f971a6-9975-45e9-b63e-231cc3527160 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.458200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 359/723] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 12 23:39:54.482759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.483497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.483784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.484053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.500376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:54.505377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:54.506393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:54.508142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 365/724] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:54.512931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.562209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:54.658347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad25957-3023-4879-acb2-e4d5a4c2f921 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.709072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.710109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.710109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.710418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.710916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:54.711235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:54.712629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:54.727738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 360/725] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:54.733750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.886207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:54.896952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0170-4e45-4615-8ca8-2452ea4bdfa5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:54.926852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:54.942755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:54.943442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:54.943768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:54.944034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:54.957957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:54.959782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:54.979828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:54.980890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:54.981782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:54.981782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:54.981782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:54.981782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 366/726] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:54.990972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.050786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.059429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ff13-37bd-49ee-9124-79f9a9b0c81e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.102383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.103045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.103352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.103621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.124059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:55.131328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:55.132316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:55.133150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 361/727] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:55.142337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.245191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659eec6-e05a-46a4-bf5e-1b1ab5fdd755 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.309044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.310005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.310466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.310884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.311857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:55.312220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14bc02-b67a-460e-ac48-bad1c7e569f8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.312539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.314279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2e4ba-c103-4660-97f1-69455f770df3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.330321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 367/728] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:55.332122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78849-a029-4955-9d77-387178972887 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.332723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78849-a029-4955-9d77-387178972887 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.333132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78849-a029-4955-9d77-387178972887 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.339536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 368/729] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:55.340896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78849-a029-4955-9d77-387178972887 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.363265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.363829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.364105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.364374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.365063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:55.365363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.368891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f672671-949b-4eca-a00c-3aeac6146266 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.383763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 362/730] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:55.385807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01762f2-a7ef-4f59-818a-9967b7f5317d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.386139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01762f2-a7ef-4f59-818a-9967b7f5317d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.386408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01762f2-a7ef-4f59-818a-9967b7f5317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.391694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.407268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e01762f2-a7ef-4f59-818a-9967b7f5317d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:55.408497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e01762f2-a7ef-4f59-818a-9967b7f5317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:55.420797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.473436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78849-a029-4955-9d77-387178972887 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 363/731] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecd284-11bb-4aae-b189-60247ed2cf61 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecd284-11bb-4aae-b189-60247ed2cf61 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecd284-11bb-4aae-b189-60247ed2cf61 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 364/732] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:55.512158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecd284-11bb-4aae-b189-60247ed2cf61 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.521653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.522376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.522649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.522914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.533260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.537405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7e1ed2b4-3974-434e-baaf-10675fcd7d25 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:55.560285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:55.561294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:39:55.562247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:55.566436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 369/733] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:55.567998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.568216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.568480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.593242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:55.594770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:55.665080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecd284-11bb-4aae-b189-60247ed2cf61 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.711253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.712317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.712317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.712595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.725385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.726027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 370/734] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:55.727911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.733070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.747307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:55.763755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:55.765779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:55.772793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:55.772793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:55.772793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:55.772793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 365/735] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:55.772793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.798969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:55.894002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ccd72-fdc3-4c29-8154-a6be70a6bde0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:55.936805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:55.937435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:55.937712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:55.937981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:55.938731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:55.939029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:55.940595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:55.965779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 371/736] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:55.966802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:55.995746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6671b-db52-4405-b8bf-d2797928e55c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.052325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.052873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.053223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.054073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.054749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:56.055053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.057304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.063465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.075824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 366/737] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:56.075824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.091731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150028b-473a-4d66-b305-2a3094b78fb6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.132314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.132957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.133233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.133501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.142723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.145476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5b2be9c2-099f-4998-945c-73f8487d8d70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:39:56.151254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:56.153166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 372/738] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:56.154911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e310f-e10f-4c0f-92aa-cf719a6d372b None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.156509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e310f-e10f-4c0f-92aa-cf719a6d372b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.156509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e310f-e10f-4c0f-92aa-cf719a6d372b None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.156930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 373/739] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:56.158180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e310f-e10f-4c0f-92aa-cf719a6d372b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.194885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73657a01-2dd9-481e-9192-3454398cd7ad None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.234292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.234923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.235209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.235511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.269826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e310f-e10f-4c0f-92aa-cf719a6d372b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.308473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.309004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.309277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.309545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.310216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.311666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.313932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.315570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.329086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 374/740] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:56.330302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.337031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.403680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.476362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 367/741] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:56.476362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.480994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40678a9-8041-45e7-b273-fcc2422fd4df None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.524830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.525492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.525763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.526036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.535627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.537588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-8167d8d5-c66d-4955-9b2c-ea38e9f7ded7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:56.552443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:56.553546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:56.554508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 375/742] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:56.556590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.556590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.556793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.580907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:56.581706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:56.627167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8bc61-96bd-4199-8e58-df0b46d04601 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.671107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.685753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.686663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.687028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.687493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.688604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:56.689066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.691499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.699627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 376/743] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 12 23:39:56.699627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.710695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 368/744] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:56.712237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.749016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:56.883439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaceff9-63ea-4712-8cf2-e4fe5a71cbae None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.907894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1122b4a-5c0e-4628-91cb-a5d79cbdbc7e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:56.930935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.931711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.932051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:56.932834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:56.933668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:56.934040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:56.936079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:56.960758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 369/745] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:56.961659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:56.997157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:56.998470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:56.998944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.000050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.026253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.030166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:57.054053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:57.056132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:57.058826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:57.058826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:57.058826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:57.067279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 377/746] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:57.067279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.102161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.134252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.145728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3fd2-fe9d-4368-9b81-76dbe2c1fbe9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.211356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d30203-6c41-4472-9fca-a70652669740 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.223500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.224441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.224973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.225391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.238002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.239961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:57.258238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:57.260162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:57.262678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:57.262678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:57.262678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:57.262678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 370/747] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:57.262678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.283408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.284956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.285530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.286023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.388156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.389532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.410273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b52d2-a145-4d4d-bcdf-5728a12bca6e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.448309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.457396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.467403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.468469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.468970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.469463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.470667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:57.471243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.473988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.495997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 371/748] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:57.497273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.556091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 378/749] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 12 23:39:57.559109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.632110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482a163-8c25-4b09-819c-bb4318b5793e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.675390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.675891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.676163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.676433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.677459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.679106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.711494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 372/750] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:39:57 2022] POST /identity/v3/services => generated 240 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:39:57.712535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.762495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e02ae-adf5-4224-8f07-86659cf44e24 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.788413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.861028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.862813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.863598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a7f7-f573-4286-8cbe-f1b5a046fab3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:57.863952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.864719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.866628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:57.867520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.870380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.885023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:57.897004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 379/751] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:57.898879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:57.929840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:57.930866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:57.931397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:57.931863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:57.945617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:57.948617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:57.969728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:57.971440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 373/752] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:57.974016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.071460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eabdb3-e59c-4b9e-b376-2dcb7fc5f813 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.117797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.118449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.118727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.118993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.132672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.135183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:58.152868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:58.154582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:58.156223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:58.156223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:58.156223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:58.156223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 380/753] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:58.156223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.190202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fca50-3a5c-4931-b485-78033a01a817 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.204742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:58.233251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.233898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.234174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.234442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.244727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.246489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.257420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:39:58.259640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:58.262359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:58.264273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 374/754] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:39:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:58.266600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.362962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b79bc-6750-453d-add2-0417448dd0f2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.391081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857f98e-a5e9-460a-a73a-f3ca5dfc3752 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.428983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.430618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.431156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.431737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.431891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.432051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.432317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.432954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:58.433252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.433629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.434634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.435564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:58.436714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.440713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.457562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 375/755] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:58.459693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.463865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 381/756] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:58.464925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.524130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:58.588001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf2f56-8883-4a8a-942f-18d3a783ee41 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.613975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:58.641130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.641730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.641988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.642285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.650179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.651708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:58.665033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:58.666772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 382/757] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:58.668457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.671729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73207d6-3115-495b-a6ec-2ad7f9626b54 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.727696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.728677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.728677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.728919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.729481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:58.729788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.731533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.762830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 376/758] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:58.772560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.890604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c64eb-7146-4b0c-bd1e-3b54222119ed None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:58.932250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:58.932876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:58.933147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:58.933431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:58.934160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=3cb125ada8d64bc49ff99383d05aa69e) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:58.934485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:58.935966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:58.942492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 383/759] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/role_assignments?user.id=3cb125ada8d64bc49ff99383d05aa69e&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:58.943850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:58.964886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:58.984957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c3b4-2750-4c64-9104-d528e6b226c4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.002586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:59.029748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.030384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.030857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.031015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.031750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:59.032021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.033496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17dd57-9a66-4278-b9d6-65d5178af1f4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.058970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 377/760] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:59.060656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c304-36d6-4611-bc6b-4bdf17401bed None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.060942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c304-36d6-4611-bc6b-4bdf17401bed None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.061244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c304-36d6-4611-bc6b-4bdf17401bed None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.073644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 378/761] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:39:59.073800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c304-36d6-4611-bc6b-4bdf17401bed None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.164390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845cd28-c03b-4c97-915b-26c925dcaad1 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.242352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c304-36d6-4611-bc6b-4bdf17401bed None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.316937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.318375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.318913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.319440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.320827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=b20995b417fc43eaac4e709042af9d24) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:59.321373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.324072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cb94c-8b70-42fa-87e5-6692edd85319 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.330898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 379/762] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/role_assignments?user.id=b20995b417fc43eaac4e709042af9d24&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:39:59.332301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.332673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.332831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.337193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.339118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.342223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.343247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.355009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:39:59.356481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:39:59.368835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.370901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:59.374463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:39:59.416399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:59.418285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:39:59.420135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:39:59.420135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:39:59.420135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:39:59.421632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 384/763] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:59.427647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.474575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 380/764] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:39:59.539822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:39:59.559060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fee5e7-621d-4490-93cc-4842ea70a4b7 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.599130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.599838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.600112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.600374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.601037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:39:59.601329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.602766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.621842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 385/765] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:39:59.630223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.770102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3d7b7-b4f8-4be7-b07f-b9c6efbb6f82 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:39:59.821071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:39:59.821741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:39:59.822010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:39:59.822283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:39:59.846029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:39:59.848938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:39:59.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:39:59.876798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:39:59.878975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:39:59.886854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 381/766] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:39:59.895107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.910134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:39:59.913464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.122921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3428-b3b1-46c8-84c2-cf891ce9a5ea None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.144414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16801e74-020b-4e9b-920b-2f66df43fdcb None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.198441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.200075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.202206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.203885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.204705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.206274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:00.220501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:00.221549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:00.222356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:00.224136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.224828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.225193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.225363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.225716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 382/767] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:00.229519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:00.229519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.229519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.233287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.256484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 386/768] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:39:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:00.268373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.288258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.396259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.438853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb87178-8220-47d9-ab04-2888d2cacd95 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.480365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.481009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.481290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.481566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.482320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:00.482633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.484250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3481d3-3bcd-4832-afc7-d3062260c151 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 383/769] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:00.492806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a911a9-10aa-4206-ad53-5f86df225ae1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.493154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a911a9-10aa-4206-ad53-5f86df225ae1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.493432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a911a9-10aa-4206-ad53-5f86df225ae1 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.495075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206e22-27a9-4f87-88e4-3511318c627b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.499383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 384/770] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:00.504557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a911a9-10aa-4206-ad53-5f86df225ae1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.557129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.558194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.558697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.559185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.561993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=97e8cd3ae37443acbb2c7c710646686a, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.564466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519c631-1e17-4eb8-a665-071d7a12ac50 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.583784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 387/771] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:00 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:00.586063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.587063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.587880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.615359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:00.616972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:00.636508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a911a9-10aa-4206-ad53-5f86df225ae1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.673838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.682328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.683421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.683908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.684409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.685268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:00.685826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.687443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.693842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 385/772] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:00.708566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:00.741610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:00.764173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 388/773] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:00.919547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d6d31-6552-4575-8d79-db6bc6b81aff None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:00.957625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:00.958621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:00.959207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:00.959715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:00.960658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] RBAC: Adding query filter params (interface=public, service_id=47397c6b631f4970805ffd6a911bfc76, region_id=RegionOne) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:00.961186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:00.962778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:00.973662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 386/774] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/endpoints?service_id=47397c6b631f4970805ffd6a911bfc76&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:00.981657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:01.159262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425c534-de99-48b6-b1ae-e66e36f58841 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:01.207459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:01.208218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:01.208437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:01.208708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:01.209391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:01.210796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:01.216362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:01.224390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 389/775] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:00 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:01.351438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:01.412282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9547cd4-5121-4436-8b5a-a9e74b30bb9d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:01.468135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:01.468836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:01.469120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:01.469486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] PATH_INFO: `/v3/services/network` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:01.470189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:01.472861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:01.490368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:01.491730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:01.492596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 387/776] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:01 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:01.500568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:01.635387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:01.717362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77da089-39e8-4511-9332-f7296e2eb423 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:01.805164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:01.806131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:01.806611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:01.807076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:01.808334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] RBAC: Adding query filter params (name=network) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:01.808868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:01.811551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b850c-20e5-4f08-8819-c14e687bc52f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:01.827064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 390/777] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:01.836380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:02.118331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db845347-5be4-40e8-b7e6-19cf551354f1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:02.168483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:02.196001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:02.197041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.197533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.198015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.199328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] RBAC: Adding query filter params (type=network) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:02.199852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:02.202495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:02.212720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 388/778] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:02.214586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794adcb-5339-4775-a4f9-0ff881759ab5 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.215056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794adcb-5339-4775-a4f9-0ff881759ab5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.215562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794adcb-5339-4775-a4f9-0ff881759ab5 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.218397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 391/779] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:02.219405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:02.224330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb337-7a53-47f5-a68b-2259d4c2e319 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.224677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb337-7a53-47f5-a68b-2259d4c2e319 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.225518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb337-7a53-47f5-a68b-2259d4c2e319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.266406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-faacb337-7a53-47f5-a68b-2259d4c2e319 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:02.271902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-faacb337-7a53-47f5-a68b-2259d4c2e319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:02.403548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 392/780] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:02.406293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974158-7b83-4cad-9a2b-ddb95adb6732 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.406977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974158-7b83-4cad-9a2b-ddb95adb6732 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.410509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974158-7b83-4cad-9a2b-ddb95adb6732 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.419720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 393/781] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:02 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:02.426168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.427379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.428165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.483286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:02.485981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:02.518607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3f68f-dd66-484f-9277-d75bc73c2e49 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:02.563242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:02.563962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.564247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.564520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.565263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] RBAC: Adding query filter params (interface=public, service_id=3ea6a818762a4da9b7ac4eb2f13b60ad, region_id=RegionOne) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:02.565599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:02.567020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:02.577979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 389/782] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:02 2022] GET /identity/v3/endpoints?service_id=3ea6a818762a4da9b7ac4eb2f13b60ad&interface=public®ion_id=RegionOne => generated 194 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:02.579548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:02.640286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:02.675741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:02.701146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eca456-f1a4-4196-adf7-e5035e9a55a5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:02.739805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:02.740560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:02.740843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:02.741115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:02.741742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:02.743173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:02.755453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 390/783] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:02 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:02.762784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 394/784] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:02.914014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.135566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0bf5e-0431-442a-b57f-175804d1dd8c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:03.161261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:03.198932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:03.200183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.200671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.201152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.221826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:03.232364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:03.234084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:03.235739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:40:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:03.247749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.347383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.479115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6858ac3-fd52-41b6-b8c9-368843c322cb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:03.561444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba08288-be89-4b26-ab1d-83e5b58f9cb0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:03.582448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:03.583085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.583377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.583647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.584393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:03.584693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:03.586114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:03.609376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 395/786] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:40:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:03.615485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.645014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:03.645679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.645958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.646235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.689390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:03.696803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:03.706419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:03.708205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:03.709778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 392/787] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:03.722434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.779339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:03.821002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14053b9b-25c9-45fd-9a78-ac6c85a6feca None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:03.858063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbfd38-4b2a-4a49-9b08-304e427e39d5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:03.887456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:03.888904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.889717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.893430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.899267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:03.899902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.900183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.900455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.901133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:03.902119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:03.903583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c5ff3-2b0d-42f0-8914-f18d92630647 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:03.914256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 393/788] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:03.917242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8aa5-8bec-4544-adb9-6df8983c6ff6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.917674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8aa5-8bec-4544-adb9-6df8983c6ff6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.918090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee8aa5-8bec-4544-adb9-6df8983c6ff6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:03.920814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:03.921593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 394/789] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:03.922876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee8aa5-8bec-4544-adb9-6df8983c6ff6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:03.925119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d9f110cf-49d0-4004-ae16-090733007f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:03.958168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:03.960190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:03.962019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:03.962019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:03.962019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:03.966686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 396/790] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:03.969514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:03.970459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:03.971274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.009157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:04.012021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:04.053825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:04.070216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee8aa5-8bec-4544-adb9-6df8983c6ff6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.131255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 397/791] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:04.131255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.151517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.152364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.152364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.152624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.171327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:04.186381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.189119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:04.243255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:04.245117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:04.246906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:04.246906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:04.246906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:04.246906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 395/792] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:04.252782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.383753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d24260-034d-48c6-b5c8-88ec5bec141e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.462337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34ed2-5c78-4043-9d66-12243c20d27f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.462901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.463588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.463866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.464137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.464803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:04.465106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.466786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ddace-26c2-4e83-80ce-5424d127de9d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.480319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 398/793] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:04.482126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585729-07e8-43a6-af5a-faebc1150a6a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.482675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585729-07e8-43a6-af5a-faebc1150a6a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.483161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2585729-07e8-43a6-af5a-faebc1150a6a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.484934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 399/794] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:04.488376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2585729-07e8-43a6-af5a-faebc1150a6a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.510675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.511060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.511375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.511646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.512313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:04.512621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.514024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b2a6-1a84-4b5e-872d-2261906501f8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.529638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 396/795] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:04.531134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.531425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.531700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.552123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:04.553440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:04.555727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:04.596429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:04.612875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2585729-07e8-43a6-af5a-faebc1150a6a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.643351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 397/796] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:04.644848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.673567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.674782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.677183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.677758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.690538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.695720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:04.715598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:04.716572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:04.717495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:04.727468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 400/797] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:04.735075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.855823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304c7d-6b17-4a12-b271-42e3dfe256d3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.890193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e6b0-39e3-45b3-8e3d-0b000d539e00 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:04.928559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.929673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.930133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.930590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.945816 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.948607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.951882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:04.952438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:04.952985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:04.953469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:04.953908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:04.954899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:04.955383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:04.957984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:04.973054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:04.974048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:04.975097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 398/798] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:04.976422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:04.991114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 401/799] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:04.992514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.002818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.137406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967264c-ddd5-4829-838e-c958f59b4987 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.140755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482683-54af-4b2a-8bc8-0d3e64368cce None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.183618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.184294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.184580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.184849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.185464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.186953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:05.197024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:05.198092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 12 23:40:05.199014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:05.203212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 402/800] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:05.204312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.210741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.211900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.212395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.212868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.228171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:05.234198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:05.235188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:05.236459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:05.237092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 399/801] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:05.237899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.307288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.324134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.347565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49ff45-69dd-4937-8bdd-b7029cf6b195 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.383100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adb707-afee-49ea-8156-fef21b24fce4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.389818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.390571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.391309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.391587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.392350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:05.392647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.395025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.426204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.426904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 403/802] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:05.427069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.427069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.427964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.428104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.435812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.437296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:05.464577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:05.465766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:05.466854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:05.466854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:05.466854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:05.471405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 400/803] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:05.472487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.550052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7605f7-aea0-494c-97f7-9476cdee4c1a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.594187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.607886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.608642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.609085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.609525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.610527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:05.611023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.613253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.616104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc30075-b5f0-40ae-9b26-6d7fae3a6d84 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.621067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 404/804] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:05.622431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.637926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.672294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.673295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.673730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.674171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.676167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:05.677039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.679679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.702800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 401/805] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:05.702800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.813484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78364-31bc-4a89-bf01-1d17757ed91e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.846655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b98f-a849-4e2d-93ed-d08a1b130167 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:05.854287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.855095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.855586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.856006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.871195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.872969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:05.890686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:05.892852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:05.894648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:05.894648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:05.894648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:05.894986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:05.894986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:05.894986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:05.894986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:05.896014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 405/806] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:05.896227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:05.896227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:05.896618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.897146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:05.915897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 402/807] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:05.917311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:05.969208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:05.969603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.055613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64770-b33e-4af6-892e-9aa094b6271e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.073526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab79607-2fd3-4f71-aad6-08652ab43d51 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.105338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.106321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.106870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.107451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.108814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:06.109357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.111956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d5a5-9932-454b-9ed8-4552ec70fe94 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.112958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.113610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.113888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.114157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.123395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 406/808] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:06.125729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021aa7e-d457-41a8-8bce-2c63eb6cfdea None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.126857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021aa7e-d457-41a8-8bce-2c63eb6cfdea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.126857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021aa7e-d457-41a8-8bce-2c63eb6cfdea None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.129447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.129647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 407/809] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:06.131031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021aa7e-d457-41a8-8bce-2c63eb6cfdea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.131840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:06.148986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:06.150210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:06.151102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:06.151102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:06.151102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:06.151102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 403/810] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:06.151637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.252263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.274895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e9c09-254b-46be-93fb-1b2e26e2d12b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.294015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021aa7e-d457-41a8-8bce-2c63eb6cfdea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.309842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.323515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.324184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.324620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.325088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.336474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.338894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.347809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.348498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.348826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.349190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.349961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:06.350327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.351868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda1f23-aeec-4a37-b546-96ed6d9e3c92 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:06.353008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:06.354684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 404/811] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:06.356505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.373747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 408/812] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:06.375770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.376219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.376637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.400107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:06.401340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:06.482291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 409/813] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:06.483176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.488818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcd63-2de9-4127-88f7-65fbb257cb12 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.533714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.534820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.535317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.535966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.537192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.539831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.573921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.591310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 405/814] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:06.591310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.624786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.631432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4ccec-f97b-46d8-9021-c2a8c0d172da None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.710975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.712120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.712630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.713181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.714336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:06.714886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.717500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.739571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 410/815] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:06.741204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.817487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0496d6-6ebc-46d1-a6f5-5a22cd030038 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.864110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.864642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.864917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.865770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.866862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:06.867264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.868770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.885880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 406/816] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:06.887302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:06.898546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a47811-b773-4b34-bc5b-a7482ae992e8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:06.955631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:06.956718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:06.957213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:06.957688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:06.969888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:06.972759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:06.995352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:06.996141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:06.996993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:06.997891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:06.998315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 411/817] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:06.999757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.029049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:07.057603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed671311-0155-460d-9c6c-b259a6505842 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.103681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.104342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.104622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.104992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.113997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.115707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:07.131065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:07.133149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:07.135818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:07.137448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 407/818] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:07.138247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.168719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de33bb8-24fe-4fde-94ab-7619051734fe None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.207060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.207794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.207999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.208272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] PATH_INFO: `/v3/services/network` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.208938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.210398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:07.220539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:07.221606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:07.222425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 412/819] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:06 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:07.223198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.272466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754302-3bc8-4bdc-a96c-a52a67f2c56f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.297040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:07.324913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.326061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.326559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.327054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/b20995b417fc43eaac4e709042af9d24/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.344491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:07.354767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=b20995b417fc43eaac4e709042af9d24, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.356319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.372724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77cf48-a8f6-4be9-b6ce-0feb4257db61 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.383685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 408/820] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:07 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/b20995b417fc43eaac4e709042af9d24/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:07.383685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.452412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.453171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.453289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.453552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.454285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:07.454585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.456170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.471665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 413/821] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:07.472844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.538210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadfbb-ad3a-4730-8613-d9ed56fea00b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.592982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.594554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.595105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.595578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.596738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:07.597213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.599809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.601236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d61f-36bd-4a8e-9634-879ad4fb3293 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.631606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 409/822] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:40:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:07.631606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.658565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.659285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.659570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.659838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.660592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] RBAC: Adding query filter params (name=network) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:07.660895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.662429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.670864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 414/823] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:07.672597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:07.672749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.677020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:07.801239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf508187-0eb5-45da-8568-058a6516256c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.846252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6709dde-ba18-489f-a4f8-5a3e292e4706 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:07.877018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.878222 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.879189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/3cb125ada8d64bc49ff99383d05aa69e/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:07.884005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=3cb125ada8d64bc49ff99383d05aa69e, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:07.886646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:07.914622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 410/824] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:07 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/3cb125ada8d64bc49ff99383d05aa69e/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:07.916346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:07.917929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:07.919878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:07.920765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:07.921656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:08.094149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:08.098003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:08.102673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:08.131781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:08.158630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7595f-6941-4f90-a99a-b0e9ca3a5655 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:08.302773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:08.305379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:08.305957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:08.306504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:08.308591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] RBAC: Adding query filter params (type=network) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:08.308591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:08.310690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:08.342748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 411/825] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:08.353903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:08.357589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 415/826] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:08.368232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:08.680955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9202a-51c5-4880-9ec6-35ea5dcc2803 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:08.689420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219e0c-400c-4aea-8880-33d4cf3118bd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:08.690598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:08.761579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:08.763154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:08.763834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:08.764484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:08.765874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:08.768550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:08.796949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:08.797737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:08.798075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:08.798393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:08.810872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 412/827] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:08.818083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:08.821441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:08.851566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:08.853369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:08.855711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:08.855711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:08.855711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:08.859067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 416/828] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:08.867167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:08.911862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:09.176757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1c137-e681-4d9f-b0e4-7951d2c77349 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:09.231154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:09.232020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:09.232394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:09.232852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:09.233673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:09.234065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:09.235628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:09.259180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 413/829] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:09.273106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:09.462525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69204fd-4198-4f6a-aae2-d931992b88cc None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:09.513027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:09.513799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:09.514207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:09.514485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:09.526531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:09.528578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-26bde0f5-457d-448f-b77c-fd5bb2848f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:09.545427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:09.547386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:09.549184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 417/830] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:09.556610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:09.736621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:09.751676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b453a-7dbb-4110-8e4d-45767743dd71 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:09.753279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b453a-7dbb-4110-8e4d-45767743dd71 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:09.753944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b453a-7dbb-4110-8e4d-45767743dd71 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:09.757282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 418/831] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:09.767424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:09.768351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:09.769063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:09.800700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:09.803444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:09.821486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:09.825134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa3d8a-3458-4546-bbbc-4751dde0614b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:09.885459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217a87-9573-405e-b56f-e13626117636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:09.886323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217a87-9573-405e-b56f-e13626117636 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:09.886806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217a87-9573-405e-b56f-e13626117636 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:09.887310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217a87-9573-405e-b56f-e13626117636 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:09.888611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217a87-9573-405e-b56f-e13626117636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:09.889140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217a87-9573-405e-b56f-e13626117636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:09.892305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217a87-9573-405e-b56f-e13626117636 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:09.919350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 414/832] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:09.926167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 419/833] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:09.928672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217a87-9573-405e-b56f-e13626117636 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:10.050701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217a87-9573-405e-b56f-e13626117636 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:10.078366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:10.092143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:10.092758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.093030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.093301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.094039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:10.094311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:10.095775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:10.102830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 415/834] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:09 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:10.292808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:10.427860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db3029-9d46-4496-b3c6-af1dba9f4674 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:10.492298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:10.497520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.497520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.497520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.497520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:10.498091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:10.516403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:10.517464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:10.518349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:10.521863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 420/835] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:10.531383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:10.642821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1df9a-36c4-41cf-b48c-5da27a1d8309 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:10.686296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:10.686987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.687446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.687763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.688500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:10.688897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:10.690405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b33782-d323-4136-a8d3-784081bcb4f1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:10.697264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 416/836] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:10.706149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:10.726711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:10.904795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58292e-8c1e-4e25-915a-c3fa1a341ec5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:10.973018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0272-d400-40bc-9861-398dc9df8e5f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.973373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0272-d400-40bc-9861-398dc9df8e5f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.973738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0272-d400-40bc-9861-398dc9df8e5f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.974300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:10.975419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 417/837] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:10.975832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.976481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.977129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.978428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:10.979113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:10.981172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736405-749c-453d-93a0-b1706c8582b5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:10.981554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736405-749c-453d-93a0-b1706c8582b5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:10.981921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6736405-749c-453d-93a0-b1706c8582b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:10.982704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fabfe1-65c1-475c-9654-1eb7b5521e34 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:10.992583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 421/838] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:11.002764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b6736405-749c-453d-93a0-b1706c8582b5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:11.004259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b6736405-749c-453d-93a0-b1706c8582b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:11.065222 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:11.092496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 418/839] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:11.272147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbc2c-a2e0-4b34-92f9-913bbb6bc467 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:11.272596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbc2c-a2e0-4b34-92f9-913bbb6bc467 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:11.272933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbbc2c-a2e0-4b34-92f9-913bbb6bc467 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:11.275405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 422/840] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:11.283735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:11.284036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:11.284372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:11.361786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbbc2c-a2e0-4b34-92f9-913bbb6bc467 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:11.443154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:11.444139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:11.448782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:11.483619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbbc2c-a2e0-4b34-92f9-913bbb6bc467 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:11.521828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 419/841] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:11.522381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:11.523363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:11.523854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:11.524346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:11.537424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:11.542712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:11.543925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:11.544764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:11.547466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 423/842] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:11.557864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:11.664794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2c936-d6a3-4136-8ff4-4913c4a9dabd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:11.681722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:11.706711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:11.707365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:11.707670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:11.707938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:11.708622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:11.708920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:11.710381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:11.720193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 420/843] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:11.725866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:11.810151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:11.971272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3d5c-d396-4e74-bbff-1ef42738b9ec None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.025707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528a6a-af6f-4686-bc5c-f9a0f582970a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.026973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:12.037437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.038868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.039587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.040273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.076556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:12.085424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:12.106794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:12.108768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:12.110680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:12.110680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:12.110680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:12.110680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 424/844] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:12.116957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.164108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.165610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.166088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.166770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.212340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:12.234896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:12.236795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:40:12.238037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:12.242098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27a30-4607-4062-a8fd-a6b9706db66b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.260785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 421/845] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:40:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:12.262767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.285652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.288486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.288886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.289219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.289975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:12.290341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:12.291971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:12.308947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 425/846] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:12.320525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.344015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:12.451970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:12.550862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153fd12-5711-4cfd-abd1-321f20db6678 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.600704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585a174-178d-459d-a996-cfaca43e00e5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.622432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.623979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.624438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.624858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.625996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:12.626491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:12.629942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:12.651257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 422/847] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:12.655435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.679933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.681399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.682183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.683139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.696062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:12.699167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:12.725134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:12.727181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 12 23:40:12.729190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:12.733439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 426/848] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:12.742790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.835378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:12.838536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be5922-7573-4deb-8797-18f3297aa8f5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.878575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:12.879137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:12.879659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:12.879833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:12.892124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:12.893669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:12.910839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:12.912562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:12.914210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:12.914210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:12.914210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:12.914210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 423/849] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:12.925370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:12.943253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7408-6efd-468f-90a2-3d77e66e6172 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:12.948041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:13.004662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:13.005707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.006074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.006485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.007773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:13.008229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:13.010640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:13.050887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 427/850] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:13.074060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:13.147356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb82c5-36a2-42b9-bc70-2af1ac717c38 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:13.219395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:13.220348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.220844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.221432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.222661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:13.223201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:13.225889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:13.259287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 424/851] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:13.262105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:13.284191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:13.294363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348357f-cf6a-4836-8b3c-cb14f331c5e6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:13.360145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:13.361286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.361935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.362417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.419029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:13.464602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c149-f86c-46da-a18a-477e8662ad64 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:13.497925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:13.500154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b003c36-39ae-496e-bc15-fbd60988db58 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:13.520396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:13.521299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.521676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.522099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.531836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:13.534356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-76614570-a226-4f98-892e-d3f077b1ec72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:13.555165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:13.562468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 425/852] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:40:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308e034-72be-48ba-a5b9-261767370cb9 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.564103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308e034-72be-48ba-a5b9-261767370cb9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.565176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308e034-72be-48ba-a5b9-261767370cb9 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.565176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 426/853] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:13.565176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1666f-18ca-44cb-a1f1-e4fa89bf6515 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.565176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1666f-18ca-44cb-a1f1-e4fa89bf6515 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.565530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1666f-18ca-44cb-a1f1-e4fa89bf6515 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.567660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 427/854] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:13.569824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1666f-18ca-44cb-a1f1-e4fa89bf6515 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:13.659315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 428/855] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:13.659315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b0ba9-e1a4-436e-940d-1a6b575e7e12 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.659315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b0ba9-e1a4-436e-940d-1a6b575e7e12 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.659315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b0ba9-e1a4-436e-940d-1a6b575e7e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.673725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:13.686724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-879b0ba9-e1a4-436e-940d-1a6b575e7e12 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:13.688718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-879b0ba9-e1a4-436e-940d-1a6b575e7e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:13.725587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1666f-18ca-44cb-a1f1-e4fa89bf6515 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:13.760646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:13.770897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:13.771838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.772221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.772647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.773542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:13.773983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:13.775734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:13.775944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 429/856] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:13.777747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:13.778162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:13.778562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:13.808995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:13.810229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:13.828616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 428/857] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:40:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:13.828616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:13.919587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 430/858] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:13.919587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:13.963293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a1cc7-68e9-4d7c-b8d2-e6d637439564 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.014378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.029853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.030927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.031435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.031910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.053346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.059290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.063382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326ebdd-921f-4196-b098-1c58dfb1f4df None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:14.091176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:14.092588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:14.093445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:14.093445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:14.093445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:14.096879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 429/859] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:14.105123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.118905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.143790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.144916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.145473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.146285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.283489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.285737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.293107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135733d-49f5-47c9-a7a2-4552a55bb675 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.337956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.338903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.339257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.339686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.340746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:14.341235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.343563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.368718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 430/860] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:14.368718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.432062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 431/861] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:14.432386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.433048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.463668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.513164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50776029-b4f1-4df3-a51c-64964c32b55c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.554928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.555814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.556189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.556621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.557438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.558883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5bf9-06d0-4f1b-b77e-2c90a68eea05 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.559079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.578259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 431/862] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:14 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:14.579504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.621924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.622861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.623297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.623705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.624616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.626892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:14.644760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:14.646326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:14.648253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 432/863] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:14.649984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.732838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.736045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ff4d-ec6d-4bfa-9501-f241801fe803 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.797727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.798744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.799175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.799652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.802356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:14.812745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.815201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:14.835396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:14.837194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 432/864] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:14.839138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:14.843920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9a26-1098-421b-ac98-6ef0dc435caa None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:14.914586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:14.915470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:14.915900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:14.916296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:14.932700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:14.934794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:14.953820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:14.955740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:14.959063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:14.959063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:14.959063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:14.959063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 433/865] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:14.959063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.019984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c30c8-dbb3-452c-87e3-c572b64e3f3d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.077633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.078461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.079093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.079538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.080934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:15.081384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.084193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.099116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 433/866] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:15.099116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.140096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40384bd-f2e4-42e2-a42a-cd1f2aa22597 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.147053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:15.181924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.182749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.183194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.183637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.184726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:15.185159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.187079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.189711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:15.217237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 434/867] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:15.217237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.296781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7b661-aed1-4e84-af1e-01c4029c4ff9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.356703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.357299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.357690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.358118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.359160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:15.359540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.361476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.375375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c2bb-b333-4722-8b5c-c3678b029fa2 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.388161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 434/868] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:15.388161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.434011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.435053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.435524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.435965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.436983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:15.437476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.439669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.447637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 435/869] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:15.449014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.544568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:15.569110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:15.579666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0a1d1-ff6e-4b91-9155-7ecca904c2a4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.647824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.648699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.649145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.649732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.650959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=3cb125ada8d64bc49ff99383d05aa69e) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:15.652035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.654457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.663421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6611e5b-f225-41fb-997f-49238b0722ca None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:15.671275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 435/870] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:15 2022] GET /identity/v3/role_assignments?user.id=3cb125ada8d64bc49ff99383d05aa69e&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:15.734949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:15.735980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:15.736413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:15.736851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:15.749738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:15.752251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:15.772622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:15.774299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:15.776167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 436/871] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:15.783963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:15.978410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f546674-1820-4497-83b6-7e57e539f085 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:16.043961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:16.044638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:16.045067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:16.045492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:16.046400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:16.046770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:16.049032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6df5c8-a7c7-406f-9893-b350d2a73742 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:16.080158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 436/872] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:16.089650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:16.259492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091bd1-f857-46c2-949b-b50020285c3e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:16.278133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:16.296724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:16.297349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:16.297618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:16.297885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:16.298623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=b20995b417fc43eaac4e709042af9d24) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:16.298924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:16.300419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393c72-20c1-4c14-913d-c29a0f00b2e2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:16.307817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 437/873] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:16 2022] GET /identity/v3/role_assignments?user.id=b20995b417fc43eaac4e709042af9d24&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:16.932273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762b8b-c0ad-4705-9dde-ea1934afb27c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:16.932601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762b8b-c0ad-4705-9dde-ea1934afb27c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:16.932933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1762b8b-c0ad-4705-9dde-ea1934afb27c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:16.934554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 437/874] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:16.940873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:16.941172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:16.941492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:16.963442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:16.964685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:17.042405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 438/875] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:17.307747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1762b8b-c0ad-4705-9dde-ea1934afb27c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:17.319672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:17.506549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1762b8b-c0ad-4705-9dde-ea1934afb27c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:17.583080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:17.584869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:17.585659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:17.586422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:17.587781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:17.590642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:17.608952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:17.610914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:17.612618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:17.617920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 438/876] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:17.623775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:17.739127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8dfc8-cf02-4d4d-89bb-1f9bc0f865bf None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:17.753951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:17.777068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:17.777690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:17.777963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:17.778226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:17.778860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:17.779154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:17.780581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:17.786177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 439/877] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:17.791792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:17.899675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:17.962163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14bf5-8cd4-4493-bb3e-292cb3d2ed01 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:18.021789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:18.022808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.023322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.023793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.024939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:18.025465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:18.028187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af2c86-ac44-4911-84f7-2999a66a3fff None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:18.041220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 439/878] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:18.047299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:18.233426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4582b-85bb-49ee-975e-a7c020243ba9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:18.271314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:18.271953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.272225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.272490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.273363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] RBAC: Adding query filter params (interface=public, service_id=53f36d875d3f4fbc89b429b68659ac07, region_id=RegionOne) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:18.273480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:18.274882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:18.275780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c0354-1758-4849-864f-7fb3fc36b8a5 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.276254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c0354-1758-4849-864f-7fb3fc36b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.276718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c0354-1758-4849-864f-7fb3fc36b8a5 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.283424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 440/879] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:18.283628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 440/880] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:18 2022] GET /identity/v3/endpoints?service_id=53f36d875d3f4fbc89b429b68659ac07&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:18.289026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf8f3-5712-4799-b70b-33233c707e90 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.289529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf8f3-5712-4799-b70b-33233c707e90 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.290074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf8f3-5712-4799-b70b-33233c707e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.290229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:18.325418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4e3bf8f3-5712-4799-b70b-33233c707e90 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:18.330349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4e3bf8f3-5712-4799-b70b-33233c707e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:18.340641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:18.419951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28528e-878c-4c0b-9b03-d683991edbf3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:18.461117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 441/881] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:18.468810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:18.469437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.469709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.470008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.470589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:18.472088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f495eb3-7302-4f07-8e40-b02832b4d5e2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:18.479116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 441/882] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:18 2022] GET /identity/v3/services => generated 1011 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:18.621570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37459e9-5940-4117-890e-05fdc19bcc31 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.622033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37459e9-5940-4117-890e-05fdc19bcc31 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.622398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37459e9-5940-4117-890e-05fdc19bcc31 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.624446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 442/883] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:18.630725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:18.631575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:18.631575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:18.653078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:18.654585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:18.741402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 442/884] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:18.960451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37459e9-5940-4117-890e-05fdc19bcc31 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:19.016210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:19.061178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:19.105266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:19.157415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37459e9-5940-4117-890e-05fdc19bcc31 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:19.170708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3277-45fc-4b67-83b5-2fffbc9a3c8e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:19.198846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:19.200190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.200667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.201198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.202060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:19.203767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:19.214732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:19.215785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:19.216603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:19.219642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 443/885] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:19.231139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:19.242561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:19.243694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.244158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.244625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.246118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:19.248752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:19.283344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 443/886] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:19 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:40:19.388233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856327d-8a40-483c-8bdb-44cbef42c9f7 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:19.427241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:19.446315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:19.447408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.447882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.448344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.449196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:19.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:19.451396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfadf0-d139-41d8-97ef-e9bb65067c78 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:19.457634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 444/887] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:19.466833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:19.627106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:19.642043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a609-b72e-42e1-93e2-00f6798a5068 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:19.709114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:19.710667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.711492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.712230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.713728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:19.714603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:19.717788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78837010-77e1-40fe-ad60-7a0f22e85144 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:19.728185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 444/888] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:19.847943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfee4a9-402e-4970-b403-198f61b9a7e6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.848335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfee4a9-402e-4970-b403-198f61b9a7e6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.848493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfee4a9-402e-4970-b403-198f61b9a7e6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.850200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 445/889] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:19.856884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:19.857696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:19.858351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:19.883442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:19.884678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:20.019930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 445/890] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:20.284934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfee4a9-402e-4970-b403-198f61b9a7e6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:20.412286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfee4a9-402e-4970-b403-198f61b9a7e6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:20.450724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:20.451415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:20.451691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:20.451984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:20.465160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:20.470214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:20.471407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:20.472747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 446/891] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:20.481670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:20.668298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:20.696491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc67e68-90d3-4232-9255-90c708e29904 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:20.766013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:20.767554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:20.768352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:20.769139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:20.770710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:20.771569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:20.774608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:20.792364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 446/892] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:20.807338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:20.870354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:20.976234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60543d-02ea-41d2-9385-c0d2f14711b0 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:21.046718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:21.048418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.048971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.049763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.075828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:21.077879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:21.107895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:21.108890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:21.109740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:21.109740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:21.109740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:21.110431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 447/893] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:21.116152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:21.243421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd5cc-7e2d-4cf7-ad16-0ea8190a7ea7 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:21.304305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:21.305476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.305940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.306418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.307682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:21.308236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:21.310840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811940b-81d3-4cd0-abfd-98ba178ec51b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:21.335174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 447/894] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:21.340631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:21.522844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:21.529938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e8902-39f5-4231-822a-5c711f5e10e2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:21.603734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:21.604822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.605565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.605796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.607080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065248-5088-4584-9fa0-641bc68ac1c9 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.607398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065248-5088-4584-9fa0-641bc68ac1c9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.607656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065248-5088-4584-9fa0-641bc68ac1c9 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.609334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 448/895] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:21.615192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.615580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.615831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.621606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:21.624449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:21.636734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:21.638089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:21.641391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:21.646428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:21.647479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:21.648419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:21.649860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 448/896] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:40:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:21.657113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:21.791711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 449/897] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:21.843591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2fd6e-003d-4b0a-bef9-6c303bf78feb None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:21.910579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:21.912379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:21.913185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:21.914100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:21.915848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:21.916793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:21.919835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:21.956626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 449/898] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:40:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:21.965638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:22.169076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab222c0-dd0a-4bd3-b130-c9e5068f70ff None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:22.236254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:22.237756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.238551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.239328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.271120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:22.339398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:22.375830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:22.378737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f3077-f097-45d5-baf7-9c636427b7bb None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:22.386001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:22.564510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e2c08-e346-4431-9e30-217f1c388a43 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:22.583273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 450/899] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:21 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:22.583273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fcf-7a42-444e-972f-d53e7e53575e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.583273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fcf-7a42-444e-972f-d53e7e53575e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.583273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25fcf-7a42-444e-972f-d53e7e53575e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.583273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 451/900] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:22.585506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44189c-c7e4-44ea-8db1-da6de3d5f15c None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.585694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44189c-c7e4-44ea-8db1-da6de3d5f15c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.586636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44189c-c7e4-44ea-8db1-da6de3d5f15c None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.588624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 452/901] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:22.590070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44189c-c7e4-44ea-8db1-da6de3d5f15c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:22.630814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:22.632027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.634971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.635318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.636303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:22.638758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-3f77c472-14de-4d6c-a376-2e3445207350 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:22.652480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:22.654303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:22.656532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 450/902] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:22.658735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1202-03be-480a-a082-90f490b1457e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.658735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1202-03be-480a-a082-90f490b1457e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.658735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e1202-03be-480a-a082-90f490b1457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.664409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:22.680781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-158e1202-03be-480a-a082-90f490b1457e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:22.682013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-158e1202-03be-480a-a082-90f490b1457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:22.758329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44189c-c7e4-44ea-8db1-da6de3d5f15c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:22.764850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 451/903] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:22.766534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.767004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.767441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.789447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:22.790952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:22.801830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:22.830154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:22.831194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:22.831696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:22.832168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:22.851854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:22.854413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:22.874124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:22.876295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:22.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:22.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:22.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:22.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 453/904] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:22.878700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:22.903594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 452/905] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:22.915008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.026294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.043592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c818d6-a83a-4893-ad46-89d7af987d2e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.054162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44f4f8-ad55-4759-9966-62dfa8f8c310 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.100676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.101269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.101393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.101563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.101837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.102541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:23.102778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.102934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.103406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.103874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.104195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.104861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:23.105332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.106926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.113256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 454/906] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:23.114118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.118061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 453/907] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:23.121871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.125666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.256070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382ab1-39cd-43e0-a952-ba90d1fd9059 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.276327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48b797-abd6-4372-9b15-ad358846f6de None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.294169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.295092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.295596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.296076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.296851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:23.297296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.298833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.305782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 454/908] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:23.306946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.339943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.340597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.340876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.341144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.341780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.343153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:23.352811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:23.353780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:23.357349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 455/909] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:23.359318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.383305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.404066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.443336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d791-5750-4438-b88f-7eeccb07bd51 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.486208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.487339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.487822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.488286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.498024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.499543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.505864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4048-2bb8-49c4-ab58-733c3fe66d62 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:23.511932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:23.521627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 455/910] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:23.522948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.561360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.562303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.562747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.563085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.564254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.566515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.596098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 456/911] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:23.597195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.665877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10140949-579b-4488-9642-f14d6470dbfa None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.701245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.711123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:23.734276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.735185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.735683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.736112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.736983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:23.737456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.739130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.739962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc785c-2518-4e36-9e3d-8b9fb0a80497 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:23.747791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 456/912] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:23.749217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.789262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:23.790030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:23.790489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:23.790850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:23.791714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:23.793415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:23.812587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 457/913] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:23 2022] POST /identity/v3/services => generated 241 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:23.813315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:23.971989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac74be-87cd-43fc-a042-9eccf0c2d80a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:24.027364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:24.028106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:24.028442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:24.028884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:24.030100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:24.030458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:24.032859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:24.044585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e19a9-2fcf-4ea9-8266-9a97b6237809 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:24.048501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 457/914] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:24.058834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:24.109870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:24.111391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:24.112164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:24.112901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:24.114052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:24.114735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:24.117408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:24.121983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:24.123429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:24.129561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 458/915] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:24.139398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:24.193932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a4c29-57a1-47de-9d0d-f8ee6aa1d48c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:24.259707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:24.260721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:24.261190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:24.261653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:24.262593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] RBAC: Adding query filter params (role.id=885fdf78f15b4ad8a461242a11d0d56a, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=b20995b417fc43eaac4e709042af9d24) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:24.263098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:24.264796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:24.272224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 458/916] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:24 2022] GET /identity/v3/role_assignments?user.id=b20995b417fc43eaac4e709042af9d24&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=885fdf78f15b4ad8a461242a11d0d56a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:24.326901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68074a3-0cec-4c7e-b95d-8e1b534f9810 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:24.400294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:24.401383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:24.401861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:24.402328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:24.403678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] RBAC: Adding query filter params (interface=public, service_id=e2db8c778480439a8b1574309d8d003d, region_id=RegionOne) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:24.404221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:24.406955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a5cca-5293-4a3f-a82b-0abe5c193894 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:24.418684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 459/917] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:24 2022] GET /identity/v3/endpoints?service_id=e2db8c778480439a8b1574309d8d003d&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:24.428325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:24.626792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda2441-7826-443c-bcd7-da88f985e2a8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:24.641278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:24.698224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:24.699161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:24.699607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:24.700019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:24.701025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:24.703439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e92e9-52ed-4cae-819d-a1483699dbcc None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:24.715597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 459/918] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:24 2022] GET /identity/v3/services => generated 1497 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:26.546247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c6e24-dd30-45f0-926c-9a8a12a18f25 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.546764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c6e24-dd30-45f0-926c-9a8a12a18f25 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.546764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c6e24-dd30-45f0-926c-9a8a12a18f25 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.548430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 460/919] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:26.554938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.555502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.555580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.577319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed81c9-d5e2-4038-8747-8cef36177c51 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.577700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed81c9-d5e2-4038-8747-8cef36177c51 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.577865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed81c9-d5e2-4038-8747-8cef36177c51 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.579509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 461/920] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:26.579796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:26.581168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.585712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:26.586320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:26.586796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:26.613662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:26.615922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:26.656116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 460/921] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:26.708875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:26.719865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 462/922] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:26.988159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.077326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.104796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:27.108059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fa516-f4e1-4368-a2ea-1f6b30fd0b09 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.147995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.148745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.149076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.149396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.150180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.151961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:27.162544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:27.163650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:27.164531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:27.167292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 461/923] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:27.176987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.194721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d5800-b95e-46ae-af31-eb5dccfbf66d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.234368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.235191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.235592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.235956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.248765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:27.253816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:27.254767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:27.255623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:27.260294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 463/924] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:27.261358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.353708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a88a7-98fc-4cba-b1a6-4033678aec64 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.369944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:27.395947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.396583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.396864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.397134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.398095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:27.398395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.399842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.406753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 462/925] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:27.415649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.452565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:27.470789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62804745-7809-42bb-ad38-a7afe7fc68b4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.520478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.521023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.521305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.521564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.522243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:27.522556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.523981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fa753-c45f-4950-91e4-b984cc6f8329 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.537116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 464/926] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:27.540701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef186-f1e9-483f-a8d1-1dd16599ccb0 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.541373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef186-f1e9-483f-a8d1-1dd16599ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.541860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef186-f1e9-483f-a8d1-1dd16599ccb0 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.546461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 465/927] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:27.548067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef186-f1e9-483f-a8d1-1dd16599ccb0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.553705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c2d11-9d5d-4082-b984-49899754f2c5 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.616787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.619042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.619801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.620514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.621964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:27.622507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.625097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7fca-1623-491f-ade0-ae9614786068 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.643008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 463/928] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:27.645302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.645871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.646342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.680068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:27.682109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:27.702142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef186-f1e9-483f-a8d1-1dd16599ccb0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.725152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:27.749878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.751078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.751585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.751993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.775746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.778466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.795336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 464/929] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:27.795336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:27.808089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:27.810163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:27.811669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:27.811669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:27.811669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:27.821312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 466/930] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:27.822381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:27.837501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:27.937508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdd783-161e-48f1-9500-73eb9911a379 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:27.981381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:27.982570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.982985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.983422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:27.984477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] RBAC: Adding query filter params (interface=public, service_id=9b9be1f1e54247fa8b4d158df8bb684c, region_id=RegionOne) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:27.984937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:27.986545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040296-673a-4529-b9b1-2244fcc9ffee None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:27.993781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 465/931] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/endpoints?service_id=9b9be1f1e54247fa8b4d158df8bb684c&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:27.995419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e51b-caee-4a0c-a3ec-57e13ee12942 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:27.995900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e51b-caee-4a0c-a3ec-57e13ee12942 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:27.996375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2e51b-caee-4a0c-a3ec-57e13ee12942 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.002876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 466/932] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:28.002876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2e51b-caee-4a0c-a3ec-57e13ee12942 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.012119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd20eb-c7f8-45ea-80dd-4d68a29a394f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.080524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.081152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.081429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.081701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.082362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:28.082669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.084197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7877e-4054-4230-9cd4-dff9188feeda None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.098582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 467/933] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:28.099910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.100194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.100562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.123822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.124043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:28.126110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:28.144707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2e51b-caee-4a0c-a3ec-57e13ee12942 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.159977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.192830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.193731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.194177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.194613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.195559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.197916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.205034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 467/934] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:40:28.205835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.247983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 468/935] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:28.247983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.348971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895c82-aec9-44ed-8aff-863bb0d8d316 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.422717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.425905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.426476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.426946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.428248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.431064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.437157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING keystone.server.flask.common [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 23:40:28.474929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.sql.core [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:40:28.474929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 23:40:28.474929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [parameters: {'id': '693ac5fc6bcf4a60911fbcee47952d22', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 23:40:28.474929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87847) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:40:28.486270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f1aee-0711-4edf-a5b3-e50d1f38a74d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 693ac5fc6bcf4a60911fbcee47952d22.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 693ac5fc6bcf4a60911fbcee47952d22. Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:40:28.491382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result.read() Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 12 23:40:28.492914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application yield resource Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise self.value Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application session.commit() Sep 12 23:40:28.494297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:40:28.496060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:40:28.497846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application result.read() Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:40:28.500912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [parameters: {'id': '693ac5fc6bcf4a60911fbcee47952d22', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.504351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:28.507214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 693ac5fc6bcf4a60911fbcee47952d22. Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.510906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 468/936] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Sep 12 23:40:28 2022] POST /identity/v3/projects => generated 262 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:28.512973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.529182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.531944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.532693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.532987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.533222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.542818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.544436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:28.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:28.559833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:28.560919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 469/937] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:28.565770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.594219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.673737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d5d5b-a6e8-4435-a0eb-bab0147dfeef None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.716722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cd908-e2bc-4219-86a9-1deb7edd240c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.739572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.740586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.740999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.741409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.753381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.756390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.763433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:28.764552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:28.765058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:28.765551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:28.766552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:28.767022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:28.768726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:28.776582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:28.777631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:28.778535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:28.783631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 469/938] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:28.785111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.802747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 470/939] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:28.804330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:28.901995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.943835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:28.962835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edadc2a-e37a-4e36-a313-57303ddeb7c3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:28.976553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71089d3-bfa5-4db0-a6c8-692e06c7698d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.021762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.022782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.023295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.023417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.023766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.024392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.024869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.024972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:29.025351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.025447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.026396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.028124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.029062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:29.041979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:29.043407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 12 23:40:29.044362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:29.045859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 470/940] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:29.047476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.057792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 471/941] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:29.059104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.232701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169610-20e6-4e46-8737-57adae3d79c9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.236811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3fd-40fb-4c40-acfa-67c3a0f56471 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.268859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:29.276806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.277880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.278612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.279112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.294115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.296191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.311595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.312773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.313321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.313883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.315175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:29.315855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:29.317273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:29.318198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:29.319098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:29.319098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:29.319098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:29.319305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.321562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 471/942] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:29.327629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:29.333648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.339347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 472/943] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:29.347974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.502442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14295-5563-49dc-8387-107cd9648045 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.562220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cd8f-641d-496b-95b5-6d0e87a5c958 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.578451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.580787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.581595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.582308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.584640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:29.585824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.589156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.626015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 472/944] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:29.632864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.633589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.634576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.635157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.635701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.636768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:29.637276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.639685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.649647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 473/945] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:29.658618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.723017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:29.786359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:29.828613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbb9a-7413-4346-b46f-3f1f2a0699b8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.858432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a535c-26d1-473f-a8ef-862d5990c0db None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:29.888663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.889674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.890123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.890624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.903594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.906142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.919136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:29.920303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:29.920752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:29.921197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:29.922314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:29.924941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:29.928299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:29.929703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:29.932407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:29.938497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 473/946] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:29.939881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:29.962690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 474/947] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:30.151783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab530df0-0d03-4213-b1b0-0d4bfbed7132 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:30.154750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:30.230922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:30.231981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:30.232470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:30.232949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:30.234229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:30.234770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:30.237564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9cd0-4272-469b-9934-07173ffb8501 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:30.261962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 474/948] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:30.271167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:30.389170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc43a35-b289-4630-926c-176aa28ff9b5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:30.413243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:30.431161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:30.432347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:30.432833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:30.433339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/b20995b417fc43eaac4e709042af9d24/roles/885fdf78f15b4ad8a461242a11d0d56a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:30.435891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=b20995b417fc43eaac4e709042af9d24, role_id=885fdf78f15b4ad8a461242a11d0d56a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:30.437603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c5be-174f-4413-8337-14e67a7a68b3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:30.455357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 475/949] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:30 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/b20995b417fc43eaac4e709042af9d24/roles/885fdf78f15b4ad8a461242a11d0d56a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:40:30.597402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd439-0af6-4711-bd5a-2bdc57a9e1d1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:30.597749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd439-0af6-4711-bd5a-2bdc57a9e1d1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:30.597941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd439-0af6-4711-bd5a-2bdc57a9e1d1 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:30.599955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 475/950] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:30.607115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:30.607115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:30.607115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:30.628674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:30.630456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:30.715504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 476/951] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:30.980351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdd439-0af6-4711-bd5a-2bdc57a9e1d1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:31.108788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdd439-0af6-4711-bd5a-2bdc57a9e1d1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:31.150573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:31.151607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:31.152031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:31.152444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:31.153407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:31.155811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:31.171611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:31.172707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:31.173548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:31.177810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 476/952] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:31.183879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:31.325232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ddb8b-19ea-4f72-be02-b67fd02bbe5d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:31.348515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:31.394324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:31.395444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:31.395896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:31.396352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:31.397524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:31.398051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:31.400692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:31.411039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 477/953] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:31.422103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:31.464859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:31.596954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a12b94-fd7c-42a7-9b68-654a5d97f3ca None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:31.646150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:31.646779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:31.647062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:31.647373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:31.648008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:31.648308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:31.649684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735ec9c-441c-44ae-97e3-e683419faccf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:31.655534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 477/954] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:31.665482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:31.829590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150d6fe-efd2-45df-83ff-bda741eed6f1 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:31.867260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:31.868126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:31.868541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:31.868964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:31.869796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:31.871384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a08c76-3803-4f6c-aa58-21266c4b7c59 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:31.898331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 478/955] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:32.405255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00775-ffa2-43a3-9657-06e405feebca None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:32.405795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00775-ffa2-43a3-9657-06e405feebca None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:32.405795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d00775-ffa2-43a3-9657-06e405feebca None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:32.407587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 478/956] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:32.413612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:32.413893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:32.414169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:32.434624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:32.435913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:32.464463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:32.472306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6a54-52d6-4df1-9409-4e9032cbd2bb None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:32.472550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6a54-52d6-4df1-9409-4e9032cbd2bb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:32.472890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db6a54-52d6-4df1-9409-4e9032cbd2bb None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:32.474398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 479/957] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:32.480633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:32.480955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:32.481175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:32.500787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:32.502363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:32.526467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 479/958] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:32.577723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 480/959] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:32.802768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:32.865628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:32.900806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:32.929252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd8b2-f844-495f-8ad6-d9fafdb556c5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:32.995485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:32.996701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:32.997177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:32.997657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:32.998870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.001813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.006706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING keystone.server.flask.common [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 12 23:40:33.028382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.sql.core [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:40:33.028382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 23:40:33.028382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [parameters: {'user_id': 'a460f9a358a547f59480c8a6dba2ddc6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 23:40:33.028382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87846) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 12 23:40:33.061061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25dbea-14d8-4459-95aa-5fda66577386 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-2395828b-154c-4ccb-a649-e5096ee9c8f3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application result.read() Sep 12 23:40:33.061297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:40:33.062783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application yield resource Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application next(self.gen) Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self.session.rollback() Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self.force_reraise() Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise self.value Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application session.commit() Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 12 23:40:33.073332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._prepare_impl() Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self.session.flush() Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._flush(objects) Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application compat.raise_( Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application flush_context.execute() Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rec.execute(self) Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 12 23:40:33.075003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 12 23:40:33.078716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application result = self._query(query) Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application conn.query(q) Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application result.read() Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 12 23:40:33.087492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a460f9a358a547f59480c8a6dba2ddc6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:33.093886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 12 23:40:33.097821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 480/960] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Sep 12 23:40:32 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c36-72fc-4473-a63c-1e07914c0c25 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c36-72fc-4473-a63c-1e07914c0c25 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.101225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0c36-72fc-4473-a63c-1e07914c0c25 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.102666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 481/961] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:33 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:33.102666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0c36-72fc-4473-a63c-1e07914c0c25 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:33.113590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.114356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.114634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.114901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.115547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.117036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.122411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-738efad1-024c-4a03-b686-fb77e0f87106 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:33.127831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:33.131394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:33.133635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 481/962] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:33.136052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.136052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.136052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.152551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:33.153768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:33.272484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 482/963] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:33.273635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:33.288245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0c36-72fc-4473-a63c-1e07914c0c25 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.304122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:33.351393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.352765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.353392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.354016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.370919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.374494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:33.402298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:33.404422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 12 23:40:33.406546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.416366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 482/964] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:33.418646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:33.467070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6ae9f-000a-4f36-9054-c618431cf350 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.539687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.541598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:33.542186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.543014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.543837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.545244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:33.545911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.549056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.566743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 483/965] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:33.568529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:33.578311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095f893-15f8-40e8-8a1c-6ed1bd28cb6f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.618232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.618745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.619054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.619404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.620119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:33.620468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.621961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.660554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 483/966] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:33.660554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:33.686880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:33.731175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce847f-dc2c-40d7-a80a-36b9d39523d6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.809175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.810495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.810976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.812040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.813380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:33.814173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:33.818601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143808-b81c-44fd-beaf-e647360a8246 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:33.828753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:33.839413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 484/967] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:33.883676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:33.884305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:33.884676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:33.884949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:33.889222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:33.902237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:33.909684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:33.911702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:33.913305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 484/968] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:33.920411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:34.089975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631c5b-352d-4354-b1e8-def82a5f3684 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:34.142197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:34.143132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.143625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.144071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.144959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:34.145497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:34.147109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:34.154039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:34.158576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 485/969] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:34.164760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:34.299918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f8432-cfbc-4160-9f23-e75fce589c27 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:34.359542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:34.360627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.361102 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.361674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.380376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:34.383023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5d5bb72c-2c9e-4e80-93f0-c707f2b14ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:34.410252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:34.411790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:34.412667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:34.412667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:34.412667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:34.415321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 485/970] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:34.422137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:34.533539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2286-97e1-4f08-871f-a60e5dab3132 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.533997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2286-97e1-4f08-871f-a60e5dab3132 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.534482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2286-97e1-4f08-871f-a60e5dab3132 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.537419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 486/971] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:34.546366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.546855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.547374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.559756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e56e5-ab5d-4248-96be-a23de9be839b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:34.572351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:34.574054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:34.599710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:34.600349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.600631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.600905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.601573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:34.601866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:34.603307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:34.619721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 486/972] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:34.624648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:34.653612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:34.691993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 487/973] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:34.703463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:34.760508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7f345-5e00-44bc-a633-521b65ce2879 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:34.814430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:34.815409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:34.815819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:34.816230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:34.828501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:34.830912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:34.851291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:34.852429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 12 23:40:34.853482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:34.854084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 487/974] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:40:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:34.861513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:34.960465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.031150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940fd8-3bab-45bb-afb3-0d323c44b67b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.096069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423214d-9679-4979-a964-e87be8bd9d1d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.098085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.099200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.099855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.100302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.101573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:35.102084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.104704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.121500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.140122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 488/975] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Sep 12 23:40:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:35.141361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.150191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.151072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.151378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.151790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.152422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.154246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:35.166651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:35.171821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:35.172702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 488/976] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:35.178711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.199993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.313337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d46a0-c071-4774-91db-ab6670ab352e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.360633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.361690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.362116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.362567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.365180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996d531-5016-4a8b-a02d-bc93ab760902 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.415383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.416160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.416451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.416743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.417634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:35.418089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.420080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.434953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 489/977] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:35.437683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.438897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.440421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.492253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.500919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.577400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b8c81-bd38-431c-83a1-775a44c9bd9a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.589266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 489/978] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:40:35.594874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.631526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.632155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.632431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.632723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.633349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:35.633647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.635050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.640808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 490/979] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:35.647206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.741665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffede57-15c8-46fc-b589-26ce44b46ede None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.786576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.787715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.788118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.788508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.793468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.808059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.810562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.813904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3d521-21c4-4c84-9e93-a7696025213a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:35.831676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:35.836711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:35.837716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:35.838630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:35.838630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:35.838630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:35.838630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 490/980] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:35.846474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.867523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:35.868759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:35.868759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:35.869046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:35.869585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] RBAC: Adding query filter params (interface=admin, service_id=9b9be1f1e54247fa8b4d158df8bb684c, region_id=RegionOne) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:35.870032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:35.871408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:35.879184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 491/981] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/endpoints?service_id=9b9be1f1e54247fa8b4d158df8bb684c&interface=admin®ion_id=RegionOne => generated 193 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:35.885344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:35.989937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678a7d2-4619-4735-8c8e-342633f9d12e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.034594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.035540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.036022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.036448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.037278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:36.037761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.040269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.041461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2a71-02ab-489f-b510-f9ea0bc686b1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.071486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 491/982] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:36.073397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:36.099895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.100430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.100698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.100956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.101573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.102951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da286f-741c-45c6-ba8e-7fd3e612ca9b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.115275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 492/983] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:35 2022] GET /identity/v3/services => generated 1497 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:40:36.158351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:36.174635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea564f8-d79b-49eb-aea1-4ba6cb5f7100 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.175005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea564f8-d79b-49eb-aea1-4ba6cb5f7100 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.175762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea564f8-d79b-49eb-aea1-4ba6cb5f7100 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.177579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 493/984] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:36.183788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.184110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.184329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.207637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:36.208910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:36.253732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13bab6-0b32-42ee-b841-72ad78fe7b5e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.257192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:36.307985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.309106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.309489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.310430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.311551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 494/985] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:36.327294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.329864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:36.345120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:36.346720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:36.348186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 492/986] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:36.352745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:36.498356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b99089-5d36-4b52-8fcb-2972aacc356f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.555998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.556919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.557333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.557736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.558831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:36.559328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.561807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.582273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:36.582554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 495/987] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:36.588661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:36.627066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:36.712061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51217c11-e4fd-489e-bc16-a59ff375981c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.717173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a16c-a739-4d0b-93d0-97e4ca6a74a8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:36.727689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:36.757042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.757657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.757929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.758194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.758924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] RBAC: Adding query filter params (role.id=885fdf78f15b4ad8a461242a11d0d56a, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=b20995b417fc43eaac4e709042af9d24) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:36.759242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.759442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:36.760438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:36.760670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a337-d8ee-4de5-a98f-7a6a418d3a21 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.760911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:36.761380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:36.762887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:36.766068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6395c-2119-4e93-9a2b-58920899fe62 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:36.779270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 496/988] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:36 2022] GET /identity/v3/role_assignments?user.id=b20995b417fc43eaac4e709042af9d24&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=885fdf78f15b4ad8a461242a11d0d56a => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:36.786835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 493/989] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:36 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:38.365827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486c7c-00d1-4526-88e2-76c279c0b0cd None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:38.366336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486c7c-00d1-4526-88e2-76c279c0b0cd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:38.366336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486c7c-00d1-4526-88e2-76c279c0b0cd None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:38.367860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 497/990] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:38.373731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:38.374363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:38.374363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:38.394072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:38.395418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:38.499650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 494/991] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:38.817194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486c7c-00d1-4526-88e2-76c279c0b0cd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:38.915491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:38.925535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486c7c-00d1-4526-88e2-76c279c0b0cd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:38.961249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:38.961886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:38.962160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:38.962424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:38.963022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:38.964426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:38.973526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:38.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:38.976262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 498/992] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Sep 12 23:40:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:38.980959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:39.084208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:39.094571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5a35-f0d3-46c5-b121-6913329ff3f3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:39.132071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:39.132696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:39.132965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:39.133226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:39.133869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:39.134255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:39.135743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:39.143986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 495/993] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:39.151332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:39.277638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6a267-3274-4d6b-a638-092cc9048ec9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:39.316125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:39.316785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:39.317065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:39.317330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:39.317968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:39.318313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:39.319695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12906ab-96fe-42a3-b3f1-dbeb06b04aa1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:39.325418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 499/994] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Sep 12 23:40:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:39.331699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:39.476137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70282a2-0c09-488a-bba5-da50ef62e0dd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:39.530041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606830e-cb0b-453b-908f-73a760682b8b None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:39.530621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606830e-cb0b-453b-908f-73a760682b8b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:39.530621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606830e-cb0b-453b-908f-73a760682b8b None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:39.532290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 500/995] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:39.535077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:39.536088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:39.536816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:39.536926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:39.538132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:39.538399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:39.538705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:39.538975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:39.539933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:39.562648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:39.564921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:39.565409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 496/996] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:40:39.590722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:39.663687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 501/997] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:39.928740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:39.950575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:40.065220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29518-ad3d-4aa8-845d-5a0592bc9f23 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:40.102789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:40.103463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.103737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.104001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] PATH_INFO: `/v3/services/image` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.104581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:40.106686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e95efe9e-7fa5-4a82-b098-cb2a3b666a05 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:40.119247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:40.120270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.121687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 497/998] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Sep 12 23:40:39 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:40.126540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:40.191609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb89e-a096-4ff8-b686-28dce99dd717 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.192053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb89e-a096-4ff8-b686-28dce99dd717 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.192370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb89e-a096-4ff8-b686-28dce99dd717 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.193736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 498/999] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:40.199430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.199698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.200117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.219799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:40.220991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:40.242215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225b29c-44ba-4ff2-992c-f06151b5b15c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:40.285178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:40.285916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.286226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.286501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.287182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] RBAC: Adding query filter params (name=image) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:40.287505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:40.288940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:40.294987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 502/1000] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:40.301107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:40.311456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 499/1001] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:40.318950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:40.417568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c72d8e-17bd-4c4b-a049-199a06c68c2b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:40.460002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:40.460490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.460765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.461031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.461657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] RBAC: Adding query filter params (type=image) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:40.461959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:40.463381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:40.468785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 503/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:40.588218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:40.605905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:40.709928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692b473-8ea5-4231-bb4d-aef6135cd195 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:40.748327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:40.748863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.749133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.749391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.749985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:40.751439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:40.761309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:40.763360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:40.765137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 500/1003] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:40.770163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:40.864293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:40.893603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2100-d685-4b1e-8748-d5e33c04d8cd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:40.950856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:40.951866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:40.952353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:40.952838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:40.953970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:40.954508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:40.957130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:40.967088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 504/1004] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:40.975921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:41.103605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:41.142134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985e2c6-077a-4039-9adc-706beb39e522 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:41.203685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:41.204740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:41.205232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:41.205726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:41.206905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:41.207534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:41.210363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:41.224855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 501/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:41.226849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:41.350530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec26ab-ec70-45cc-980c-50be43d6adb1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:41.402641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:41.403389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:41.403667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:41.403932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:41.404812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] RBAC: Adding query filter params (interface=public, service_id=2e47d3e0b983452b9fa801109a12911b, region_id=RegionOne) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:41.405233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:41.407005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a37347-9011-49b9-97b9-3d73b43ed625 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:41.414174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 505/1006] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:41 2022] GET /identity/v3/endpoints?service_id=2e47d3e0b983452b9fa801109a12911b&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:41.420836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:41.562394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842b12-e0dd-43ad-a564-de4481b61002 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:41.619107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:41.620061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:41.620494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:41.620958 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:41.622067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:41.624336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bcc48-8a05-4c1c-a13e-78a9b66c8dc7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:41.643530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 502/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:41 2022] GET /identity/v3/services => generated 1727 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:40:41.808949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23999b-9a3e-481d-a3ee-cc7a8218aebd None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:41.809511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23999b-9a3e-481d-a3ee-cc7a8218aebd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:41.809511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23999b-9a3e-481d-a3ee-cc7a8218aebd None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:41.810942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 506/1008] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:41.817883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:41.818149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:41.818423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:41.826533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:41.839869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:41.840739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:41.938251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 503/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:42.210478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23999b-9a3e-481d-a3ee-cc7a8218aebd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:42.214574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:42.346383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23999b-9a3e-481d-a3ee-cc7a8218aebd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:42.387004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:42.387940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:42.388345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:42.388758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:42.390397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:42.392741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c4bf2-450b-4d40-b313-ac54b0aa0a17 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:42.395827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 507/1010] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:42.402696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:42.580862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6597a1-24ff-42eb-a83a-3839c4ea8c70 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:42.649035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:42.650096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:42.650536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:42.650983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:42.652114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:42.654268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d1ad9-2526-4178-936c-5de38fbd78c1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:42.661177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:42.699254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 504/1011] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:42 2022] POST /identity/v3/projects => generated 311 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:42.734848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f189b-5bac-467a-9c0f-3891a881c660 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:42.735695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f189b-5bac-467a-9c0f-3891a881c660 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:42.736161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f189b-5bac-467a-9c0f-3891a881c660 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:42.738982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 508/1012] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:42.746359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca817-fc6a-4258-9782-6c4ac1cf0e6a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:42.746770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca817-fc6a-4258-9782-6c4ac1cf0e6a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:42.747110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca817-fc6a-4258-9782-6c4ac1cf0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:42.773040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-bb8ca817-fc6a-4258-9782-6c4ac1cf0e6a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:42.774710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-bb8ca817-fc6a-4258-9782-6c4ac1cf0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:42.884400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 505/1013] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:43.152087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f189b-5bac-467a-9c0f-3891a881c660 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:43.274271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:43.331402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f189b-5bac-467a-9c0f-3891a881c660 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:43.378336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:43.379001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:43.379381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:43.379644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:43.380314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:43.381714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374df50-2253-46a6-9951-c54c009ab18e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:43.408649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 509/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:40:43 2022] POST /identity/v3/services => generated 242 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:40:43.885853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e1b8-f05e-4efe-bbf6-07f18d61e9ea None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:43.886412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e1b8-f05e-4efe-bbf6-07f18d61e9ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:43.886770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e1b8-f05e-4efe-bbf6-07f18d61e9ea None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:43.889247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 506/1015] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:43.895327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:43.895683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:43.895864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:43.917552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:43.918919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:44.003257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 510/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:43 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:44.271701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2e1b8-f05e-4efe-bbf6-07f18d61e9ea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:44.388427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2e1b8-f05e-4efe-bbf6-07f18d61e9ea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:44.410354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:44.437121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:44.437747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:44.438036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:44.438306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:44.438941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:44.440364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:44.449738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:44.451748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 12 23:40:44.453530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:44.456325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 507/1017] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:44.457427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:44.521656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:44.576818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da6d7-defb-4550-850e-25675e39fef8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:44.616895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:44.617512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:44.617803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:44.618046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:44.618714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] RBAC: Adding query filter params (name=compute) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:44.619012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:44.620420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:44.626367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 511/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:44.632169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:44.788879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549400f8-23ff-4f73-8aed-3058da12fe7f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:44.849690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:44.850625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:44.851111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:44.851624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:44.852796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:44.853349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:44.855953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34098c-e22c-498c-aa94-dd25f0a1e5ff None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:44.865486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 508/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Sep 12 23:40:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:44.874713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:45.059180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f707fd-b1b2-4d0d-b24e-850880b0d6af None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:45.090641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39383a7-60a5-4e94-a99c-8ae3b445c2be None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.091242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39383a7-60a5-4e94-a99c-8ae3b445c2be None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.091242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39383a7-60a5-4e94-a99c-8ae3b445c2be None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.092787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 509/1020] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:45.098359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1a53-c011-4ace-856a-e0c42814d732 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.098681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1a53-c011-4ace-856a-e0c42814d732 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.098902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e1a53-c011-4ace-856a-e0c42814d732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.100540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:45.121195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:45.121840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.122119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.122400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.122561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b80e1a53-c011-4ace-856a-e0c42814d732 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:45.123091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:45.124657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:45.124867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b80e1a53-c011-4ace-856a-e0c42814d732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:45.153768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 512/1021] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:45.209992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:45.279284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 510/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:45.600178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:45.639205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852c14-b9bd-4fc6-bef2-390a48ed4135 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.639757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852c14-b9bd-4fc6-bef2-390a48ed4135 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.640240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852c14-b9bd-4fc6-bef2-390a48ed4135 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.643276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 511/1023] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:45.649651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.650254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.650656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.677439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:45.679018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:45.744166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae248-6206-4aef-a6c2-471f30c40664 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:45.772778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 512/1024] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:45.795817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:45.796921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:45.797316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:45.797798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:45.799614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:45.802190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:45.807270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 513/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:45.810809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:45.945252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:45.949951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a18f41-58c8-4f1e-8b2c-6daff37a9b84 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:46.001972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:46.002942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:46.003351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:46.003639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:46.004743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:46.006689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:46.040100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 513/1026] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:40:45 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:40:46.042299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:46.109742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:46.229057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e04802-ee2a-4c47-8dc7-a74db04bae49 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:46.285447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:46.286515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:46.286986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:46.287503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] PATH_INFO: `/v3/services/image` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:46.288571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:46.290999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:46.301623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:46.303473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:46.304946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 514/1027] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Sep 12 23:40:46 2022] GET /identity/v3/services/image => generated 86 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:46.311013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:46.450411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bed6-ad46-46bf-aa8b-48d0e4d74824 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:46.490063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:46.490590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:46.490868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:46.491139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:46.491814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] RBAC: Adding query filter params (name=image) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:46.492115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:46.493525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:46.499583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 514/1028] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:46.506152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:46.677375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4f7f-e96a-4320-a0ac-d8d63367afc2 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:46.696595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:46.732056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:46.732696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:46.732977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:46.733333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:46.733986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] RBAC: Adding query filter params (type=image) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:46.734290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:46.735838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:46.744214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 515/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:46.751995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:46.778113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:46.881910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387316b7-ac5f-4d48-8d6a-9cce796073a3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:46.928906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:46.929849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:46.930214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:46.930615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:46.931687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] RBAC: Adding query filter params (interface=public, service_id=6bbef721077f49ecb92a71786a72628d, region_id=RegionOne) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:46.931999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:46.933823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c81d57-3f7a-40aa-973d-959d39673208 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:46.941847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 515/1030] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:40:46 2022] GET /identity/v3/endpoints?service_id=6bbef721077f49ecb92a71786a72628d&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:46.949720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:47.076954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46c7ce-deee-41a3-a9eb-1d74ba7da26f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:47.122941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:47.123644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:47.123926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:47.124196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:47.124916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:47.126438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f01c0-3188-4a0d-b2c9-93d241f01067 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:47.134631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 516/1031] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:40:46 2022] GET /identity/v3/services => generated 1957 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:40:47.508040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a91b0-a942-42c9-9225-ae0ea4a52eb0 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:47.508040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a91b0-a942-42c9-9225-ae0ea4a52eb0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:47.508040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a91b0-a942-42c9-9225-ae0ea4a52eb0 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:47.509437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 516/1032] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:47.515598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:47.515945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:47.516171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:47.537008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:47.538268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:47.578858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:47.625433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 517/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:47.889826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a91b0-a942-42c9-9225-ae0ea4a52eb0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:47.998064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:48.017123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a91b0-a942-42c9-9225-ae0ea4a52eb0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.058249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:48.058914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.059185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.059477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.074129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:48.079027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:48.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:48.084425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 517/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:48.086828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:48.222414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cf384-7c1d-4830-a635-2e82f8f5abbc None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.271295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:48.272405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.272949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.273373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.274060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:48.274364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:48.276449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391b76-cb47-4b17-b0e4-a8638506923f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:48.289725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 518/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:48.297592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:48.429245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4333-9d77-4a5e-83b1-e447d8d6dab8 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.430527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4333-9d77-4a5e-83b1-e447d8d6dab8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.430818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df4333-9d77-4a5e-83b1-e447d8d6dab8 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.432513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 519/1036] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:48.437935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.438277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.438512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.459358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:48.460962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:48.487174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898c25a-ef9a-4284-a74a-067d80a23740 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.490693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:48.547302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 520/1037] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:48.550253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:48.551438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.551899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.552340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.571370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:48.574008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:48.578586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:48.597221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:48.599458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:48.601367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:48.601367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:48.601367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:48.601367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 518/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:48.605111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:48.724746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0084e0-b9a8-4117-a577-c01274929375 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.768602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:48.769152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.769463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.769778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.770443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:48.770740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:48.772189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:48.786278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 521/1039] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:48.795212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:48.833951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:48.870010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:48.926742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b3258-0804-4c18-9afe-7e3189dac511 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.975865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d266e-8c01-4eef-852c-ebd638e6cbbd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:48.983781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:48.984427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:48.984701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:48.984976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] PATH_INFO: `/v3/users/nova` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:48.998332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.001276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:49.023721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:49.024825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:49.025791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 519/1040] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:40:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:49.034860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.038572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.039674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.040135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.041457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.062838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:49.074779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:49.076661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:49.079025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:49.080772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 522/1041] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:49.085335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.130140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.189284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d90cf8-f874-4a93-ae02-ded847904abf None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.213235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.246291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.246757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e199a-f394-4c51-8412-4a071bb31a48 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.247323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.247763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.248192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.249562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:49.250112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.252892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.278434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 520/1042] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:49.284155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.298733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.299465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.300094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.300479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.301203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:49.301497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.302872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.321503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 523/1043] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:49.322609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.443062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dded2-d60a-4720-baa9-7f2e9ea42bc6 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.461609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dec09-d042-4364-b60e-3e28edb26978 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.462857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.493386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.494243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.494796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.495321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.507893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.508825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.509246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.509666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] PATH_INFO: `/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.524284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.528201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] RBAC: Authorizing `identity:get_user(user_id=711fbfd22c9f4b4b8eeed21ab98d7259)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.530250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.537990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 524/1044] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259 => generated 285 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:49.539649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.617092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.618758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81aaf7-5df3-4b6e-82c2-2f75b1beeb04 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.675995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97181c-f35b-4f16-8091-76794e186b9e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.717252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.718067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.718346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.718615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.743732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 521/1045] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:40:49.745053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e7a8-a99d-4bd9-962a-1b718b098d2a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.745343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e7a8-a99d-4bd9-962a-1b718b098d2a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.745603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68e7a8-a99d-4bd9-962a-1b718b098d2a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.746943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 522/1046] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:49.752257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68e7a8-a99d-4bd9-962a-1b718b098d2a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.770538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.804103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:49.807609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.809572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c56ac-c7d0-46cf-8d92-5a3837db09bf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.872462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68e7a8-a99d-4bd9-962a-1b718b098d2a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:49.912576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:49.913540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.913540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.913775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.926171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:49.927715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:49.942337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 525/1047] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:49.945930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:49.948432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:49.949381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:49.949381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:49.949381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:49.949381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 523/1048] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:49.949623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:49.949623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:49.949784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:49.955699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:49.976255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:49.977572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:50.050751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 526/1049] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:50.051956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.074017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.098531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.115984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5139200-0fdc-4a78-9bb1-e5f690a453d1 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.152251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.152767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.153038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.153305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.153948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.154245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.155662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.172538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 524/1050] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:50.173942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.181852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadd293-bf27-4d4f-8803-a2e13cf48fd5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.230146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.230828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.231046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.231353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.232474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.233904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.236055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 527/1051] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:50.241879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.290143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424dc11-aa17-4c2b-b800-9b3543469267 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.322156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.332720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.333265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.333528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.333767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.341159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.342498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:50.353974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:50.354963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:50.355894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:50.356335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.356409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 525/1052] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:50.356409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.365157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e536cd-93d1-4c61-9ab5-5ceddb172722 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.401907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.402512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.402782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.403047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.404077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71, user.id=711fbfd22c9f4b4b8eeed21ab98d7259) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.404247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.405719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.411940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 528/1053] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/role_assignments?user.id=711fbfd22c9f4b4b8eeed21ab98d7259&scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:50.412672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.485201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f328-7429-4bc8-b512-3d3366211631 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.536299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.537257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.537689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.538101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] PATH_INFO: `/v3/services/image` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.538837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.540461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:50.551488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:50.555427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:50.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 526/1054] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:50.559186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.561157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c13f3-a474-4b28-b00a-c87abe62c544 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.608301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.609070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.609353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.609621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.611129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.611129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.613029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.627910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.629949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 529/1055] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:50.636575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.671278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.695719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3964d-95df-4316-9c60-0aa7bdc25a37 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.736739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.737477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.737817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.738156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.738871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] RBAC: Adding query filter params (name=image) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.739270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.740757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.750828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 527/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:50.752033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:50.762069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca05849-834d-4946-92a3-a8cf629aeede None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.804834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.805522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.805732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.806210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.807421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] RBAC: Adding query filter params (role.id=885fdf78f15b4ad8a461242a11d0d56a, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.807740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.809294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.815977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 530/1057] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=885fdf78f15b4ad8a461242a11d0d56a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:50.885666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba0064-805c-482f-8fdc-b058549310ef None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:50.909747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:50.938850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:50.939524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:50.939800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:50.940065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:50.940738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] RBAC: Adding query filter params (type=image) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:50.941033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:50.942555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c983528-a9ac-455e-9c8c-a8c0fa0550ea None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:50.949133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 528/1058] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:50.954989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:51.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:51.105816 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e9ff-d994-4b1d-8c9b-3d2dfc43436a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:51.169132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:51.170031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:51.170578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:51.170992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:51.172166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:51.175032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4504f3-fb1b-4eb1-ad18-5e701a822680 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:51.199595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 531/1059] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:40:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:40:52.594425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38151c74-c599-41d6-9fa8-36d7e7e33379 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:52.595054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38151c74-c599-41d6-9fa8-36d7e7e33379 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:52.595054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38151c74-c599-41d6-9fa8-36d7e7e33379 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:52.599261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 529/1060] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:52.602224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5c1f-5f1a-4c65-939e-a5fd4fce64d6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:52.602770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5c1f-5f1a-4c65-939e-a5fd4fce64d6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:52.602857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b5c1f-5f1a-4c65-939e-a5fd4fce64d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:52.621792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-8e3b5c1f-5f1a-4c65-939e-a5fd4fce64d6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:52.623062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8e3b5c1f-5f1a-4c65-939e-a5fd4fce64d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:52.714784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 532/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:53.053239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38151c74-c599-41d6-9fa8-36d7e7e33379 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.165639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf08c-61bb-43c7-94cf-ab043a1b2277 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.165639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf08c-61bb-43c7-94cf-ab043a1b2277 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.165974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf08c-61bb-43c7-94cf-ab043a1b2277 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.167212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 533/1062] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:40:53.172416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4879-2999-482c-96c4-9207b7defdde None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.172724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4879-2999-482c-96c4-9207b7defdde None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.172964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4879-2999-482c-96c4-9207b7defdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.182408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38151c74-c599-41d6-9fa8-36d7e7e33379 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.192430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-13fa4879-2999-482c-96c4-9207b7defdde None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:53.193651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-13fa4879-2999-482c-96c4-9207b7defdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:53.223200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.223871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.224147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.224420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.236985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:53.241215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:53.242168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:53.243166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 530/1063] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:53.248590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.274605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:53.291272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 534/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:53.307884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:53.412234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5cfff-71c1-48c8-8f0c-95ce5b97c016 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.412439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330fff-2dde-45e5-acef-41e0c51c8e2e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.412722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5cfff-71c1-48c8-8f0c-95ce5b97c016 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.413058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5cfff-71c1-48c8-8f0c-95ce5b97c016 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.416638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 535/1065] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:53.422125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.422511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.422709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.448569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:53.449801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:53.453563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.454430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.454567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.454740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.455449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:53.455739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:53.457198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:53.467966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 531/1066] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:53.473959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.539771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 536/1067] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:40:53.560178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.595124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:53.599001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91facd09-9fd5-4800-b088-c5c8be240406 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.642737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.643412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.643687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.643957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] PATH_INFO: `/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.645161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] RBAC: Authorizing `identity:get_user(user_id=711fbfd22c9f4b4b8eeed21ab98d7259)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:53.646662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:53.653932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 532/1068] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:53.655453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.687957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876339b-2805-4e74-a1d9-1ad2f8bacc62 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.725030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:53.734328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.734994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.735299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.735569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.747889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:53.752991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:53.754087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:53.754932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 537/1069] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:53.760689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.779459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1196bc-c0ef-4854-a72e-705355785075 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.818716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.819424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.819692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.819961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.821163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:53.822709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:53.824564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 533/1070] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:53.825384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.869660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:53.885202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d1b0-e71e-438e-b5e8-fc6bf0b43a6f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.924346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.925328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.925650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.925916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.926592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:53.926893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:53.928331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:53.937379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 538/1071] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:53.938190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:53.948553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cdf41-88b1-4cc3-8dba-d50a4eb0d77b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:53.986696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:53.987360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:53.987636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:53.988280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:53.991662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.051423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa54a-111f-4c7c-a11e-18e0395fcd85 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.068018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.070083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.090539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.091232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.091524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.091795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71/users/711fbfd22c9f4b4b8eeed21ab98d7259/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.094397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0687ad6e8534878bd9ff6ab9f15ac71, user_id=711fbfd22c9f4b4b8eeed21ab98d7259, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.096062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.107317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.111322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 539/1072] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:53 2022] PUT /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71/users/711fbfd22c9f4b4b8eeed21ab98d7259/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:54.112158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.207292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 534/1073] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:54.214332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.272436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db1d9-bfe4-49df-a92e-1c9b2bc2ce50 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.286449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.327597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.328122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.328402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.328669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.341019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.342520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:54.363770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:54.364804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:54.365692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:54.365692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:54.365692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:54.365692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 540/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:54.372097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.386824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e34bf-bb77-401a-88b0-6531285b67b0 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.431410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.432362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.432606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.432887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.435890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.444825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.446658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:54.462888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:54.464067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:54.464952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:54.464952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:54.464952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:54.467615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 535/1075] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:54.470997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.499808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73782277-846f-426d-a6f4-2636087dfbb3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.543333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.543966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.544387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.544504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.545461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:54.545461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.546920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.563958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 541/1076] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:54.570295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.586838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec6b28-8c69-4b99-87da-74ec9d82d955 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.596767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.623898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.624577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.624843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.625160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.625800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:54.626096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.627546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.640219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 536/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:54.689889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604c7c9-d7ad-4232-8c0d-b96134945849 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.745574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.745858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.746068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.746448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.757618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.759124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:54.776434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:54.777482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:54.778543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:54.783750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 542/1078] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:54.785443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:54.848135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:54.911975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7f161-a661-4335-afc3-772f24c97b6d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:54.957525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:54.958611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:54.959060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:54.959527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:54.960881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:54.961417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:54.964095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:54.987055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 537/1079] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:54.993712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:55.101938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:55.129397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7a4af-6d5e-4715-ba02-b2459003f1cc None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:55.169514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:55.170169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:55.170444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:55.170713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:55.183712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:55.185190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:55.201627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:55.202801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:55.203729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:55.203729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:55.203729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:55.203729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 543/1080] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:55.209034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:55.325723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390ef57-ca7c-459e-a32d-b8f4d6167a85 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:55.333724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:55.361808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:55.362327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:55.362596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:55.362860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:55.363575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:55.363874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:55.365261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:55.377068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 538/1081] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:55.383023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:55.498850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc7fa8-bbdd-4b92-962c-723794eb5def None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:55.538184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:55.539123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:55.539574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:55.539988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:55.542686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:55.551179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:55.553675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:40:55.570553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:55.571533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:40:55.572421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 544/1082] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:55.577906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:55.680787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a285-53ba-4e85-87d0-95d10653871d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:55.715249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:55.715866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:55.716139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:55.716405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:55.717106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:55.717407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:55.718902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87265a9a-dc08-4b7f-8cc4-b69cf88da022 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:55.732847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 539/1083] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:55.739296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:55.846686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158aa32-6a1f-47f8-a36b-4c755288b640 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:55.893166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:55.893858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:55.894130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:55.894437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/885fdf78f15b4ad8a461242a11d0d56a` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:55.896881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=97e8cd3ae37443acbb2c7c710646686a, role_id=885fdf78f15b4ad8a461242a11d0d56a)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:55.899150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89660e82-f612-49a2-a4e0-16af600861be None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:55.914416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 545/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:40:55 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/97e8cd3ae37443acbb2c7c710646686a/roles/885fdf78f15b4ad8a461242a11d0d56a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:40:56.347810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5e27-c4af-4d3d-a556-3acf4e946093 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:56.347999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5e27-c4af-4d3d-a556-3acf4e946093 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:56.348246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5e27-c4af-4d3d-a556-3acf4e946093 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:56.349620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 540/1085] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:56.355389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:56.355678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:56.356283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:56.375863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:56.377078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:56.434401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:56.455188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 546/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:56.721514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf5e27-c4af-4d3d-a556-3acf4e946093 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:56.750828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:56.834349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf5e27-c4af-4d3d-a556-3acf4e946093 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:56.869181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:56.869828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:56.870091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:56.870357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:56.881298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:56.886083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:56.887033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:56.888342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 541/1087] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:56.893555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:57.000481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419347ba-101e-4c88-bd21-76b3a794e4dd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:57.036782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:57.037399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:57.037673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:57.037944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:57.038643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:57.038949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:57.040363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:57.049251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 547/1088] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:40:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:57.056743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:57.153820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:57.170245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481dcea-7e77-4f95-8d4a-2efcf4312cf4 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:57.211018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:57.211710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:57.211983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:57.212250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] PATH_INFO: `/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:57.213470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] RBAC: Authorizing `identity:get_user(user_id=711fbfd22c9f4b4b8eeed21ab98d7259)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:57.214970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:57.216748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 542/1089] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:40:57 2022] GET /identity/v3/users/711fbfd22c9f4b4b8eeed21ab98d7259 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:57.223165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:57.285581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:57.346311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4700-3d82-4783-b067-2a6dc330f46a None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:57.384373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:57.385415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:57.385909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:57.386400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:57.466190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:57.467693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:57.516142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:57.578730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 548/1090] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:40:57.584856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:57.693620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9223-f79e-4081-aab1-de1f0b53ab7c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:57.731853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:57.732499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:57.732754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:57.733019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:57.734192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:57.735676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630ddc2-cd95-4d66-82a8-ed9938a1e5b6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:57.737351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 543/1091] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:40:57 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:57.744291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:57.883137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039c51-3a52-4297-bfe6-2fa06e13b041 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:57.930648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:57.931340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:57.931531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:57.931803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:57.932547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71, user.id=711fbfd22c9f4b4b8eeed21ab98d7259) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:57.932846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:57.934279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5244e60-4734-4cfd-aacc-35d9fdf2d637 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:57.940647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 549/1092] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:40:57 2022] GET /identity/v3/role_assignments?user.id=711fbfd22c9f4b4b8eeed21ab98d7259&scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:58.098226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3309-1398-4d14-9449-8bf79dc1fb50 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:58.098500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3309-1398-4d14-9449-8bf79dc1fb50 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:58.098798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3309-1398-4d14-9449-8bf79dc1fb50 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:58.100369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 544/1093] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:40:58.107678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:58.107940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:58.108229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:58.129671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:40:58.130934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:40:58.134832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:58.231318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 550/1094] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:40:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:40:58.498244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3309-1398-4d14-9449-8bf79dc1fb50 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:58.524213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:58.625670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3309-1398-4d14-9449-8bf79dc1fb50 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:58.666098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:58.666750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:58.667024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:58.667323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:58.679922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:40:58.684898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:58.686167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:58.687567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 545/1095] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Sep 12 23:40:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:58.692511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:58.803244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290b3b1-045c-4a86-bd13-fb7f34c408ee None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:58.843673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:58.844300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:58.844571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:58.844833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:58.845495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:58.845788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:58.847199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:58.856130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 551/1096] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:40:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:58.862277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:58.958104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:58.983451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb9755-72f2-4548-8d50-b87225613683 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.021888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.022561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.022837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.023105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.035112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.036631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:40:59.052751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:59.054322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:40:59.055606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:40:59.055606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:40:59.055606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:59.055606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 546/1097] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:40:59.061748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.065360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:59.178432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed808fb-0bb0-453e-a0c2-cff5422ed101 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.216413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.217030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.217297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.217604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.218185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:59.218478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.219916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.233241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 552/1098] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:40:59.238911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.356443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534524d-7e28-484c-9239-5ca3604a563d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.391692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.392328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.392596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.392865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.395331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:59.403690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.405588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:40:59.417857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:40:59.419089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:40:59.420849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 547/1099] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:40:59.425519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.493273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:59.531622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f631-d46f-4028-b5bd-58dea115babc None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.564990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.565594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.565886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.566145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.566873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:40:59.567189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.568626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.585781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 553/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:40:59.591857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:40:59.693278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c843-ae7d-4297-9f1d-78e6828266c9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:40:59.728535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:40:59.729145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:40:59.729416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:40:59.729683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:40:59.791648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:40:59.800281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:40:59.801749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:40:59.903372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 548/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:40:59.909396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.012314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6480-acda-43c7-9861-1f51091d3a43 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.032983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.048146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.048787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.049059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.049695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.061730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.063258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-97f7280c-e65a-404b-93e9-90921a8b48d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:41:00.080614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:00.081744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:41:00.082692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:41:00.082692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:41:00.082692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:00.082692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 554/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:00.088413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.187654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4554-f310-4d09-8a5f-a7363bd47c2d None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.187961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4554-f310-4d09-8a5f-a7363bd47c2d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.188224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be4554-f310-4d09-8a5f-a7363bd47c2d None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.189737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 555/1103] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:00.191633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.195531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.195814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.196091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.200100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7678b-2bca-40f4-9921-b38cdfd0ecee None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.214534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:00.215780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:00.237416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.238147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.238421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.238689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.239367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:00.239673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.241056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.253148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 549/1104] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:00.258895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.284640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 556/1105] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:41:00.360853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4119f-a74e-4115-b8b6-9df9e6e5d331 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.397787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.398426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.398704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.398975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.405871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.407367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:41:00.418645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:00.419753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:41:00.421005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:41:00.423366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 550/1106] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:41:00.426127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.526479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e609fc-3112-42f1-8715-a7004eda4a9b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.528037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.553019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.563055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df2601-851b-4221-800d-725d897c08aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.563424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df2601-851b-4221-800d-725d897c08aa None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.563701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df2601-851b-4221-800d-725d897c08aa None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.563971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df2601-851b-4221-800d-725d897c08aa None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.564670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df2601-851b-4221-800d-725d897c08aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:00.565014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df2601-851b-4221-800d-725d897c08aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.566508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df2601-851b-4221-800d-725d897c08aa None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.570337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.578551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 557/1107] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:00.585151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df2601-851b-4221-800d-725d897c08aa None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.668963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62576-7ec9-402b-84b0-98ab083d832a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.715037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df2601-851b-4221-800d-725d897c08aa None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.717948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.718735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.719024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.719375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.720527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.722293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.723918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 551/1108] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:00.729728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:00.751052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.751691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.751970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.752239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.752965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] RBAC: Adding query filter params (role.id=885fdf78f15b4ad8a461242a11d0d56a, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=97e8cd3ae37443acbb2c7c710646686a) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:00.753267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.754763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf6a17-ea36-4b65-ad1a-1c6105daa3ff None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.761115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 558/1109] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:00 2022] GET /identity/v3/role_assignments?user.id=97e8cd3ae37443acbb2c7c710646686a&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=885fdf78f15b4ad8a461242a11d0d56a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:41:00.822483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:00.844426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c40f-9d8f-4578-b069-27a2b5a425ed None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:00.885384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:00.886011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:00.886286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:00.886524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:00.887280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:00.888756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161229d8-0aa2-406d-8e24-65b7369707da None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:00.915823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 552/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:41:00 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:41:03.129964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66efc-2250-449c-b14a-34821b5c94eb None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:03.130479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66efc-2250-449c-b14a-34821b5c94eb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:03.130479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c66efc-2250-449c-b14a-34821b5c94eb None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:03.131885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 559/1111] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:03.137574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:03.138006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:03.138095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:03.156831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:03.158004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:03.226758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 553/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:41:03.494762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c66efc-2250-449c-b14a-34821b5c94eb None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:03.523985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:03.602796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c66efc-2250-449c-b14a-34821b5c94eb None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:03.638394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:03.639046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:03.639349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:03.639615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] PATH_INFO: `/v3/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:03.641336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] RBAC: Authorizing `identity:get_role(role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:03.642744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:03.644169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 560/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:03 2022] GET /identity/v3/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:03.650655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:03.731843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:03.770361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e62533-bda6-409b-9af6-40ff84618135 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:03.806526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:03.807158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:03.807458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:03.807719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] PATH_INFO: `/v3/users/b4ea621d5c894214a317b072289e756c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:03.822515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] RBAC: Authorizing `identity:get_user(user_id=b4ea621d5c894214a317b072289e756c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:03.824300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:03.826187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 554/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:03 2022] GET /identity/v3/users/b4ea621d5c894214a317b072289e756c => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:03.833860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:03.930891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:03.938005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a590ea6-c090-4e87-9927-724ab70695a1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:03.970945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:03.971572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:03.971846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:03.972137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:04.040548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:04.041926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:04.138184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:04.146506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 561/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:04.153893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:04.262501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4dfa5-0bd1-4d0f-9074-4e09bfa5e1ab None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:04.288777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:04.298718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:04.299700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:04.299979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:04.300277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:04.301390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:04.302838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268a94c-f8f9-4d9f-aebe-058a7fce854b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:04.304518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 555/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:04 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:04.314673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:04.420911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242161e6-651f-4221-88ac-38acf047ff1e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:04.458249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:04.458859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:04.459135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:04.459440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:04.460165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71, user.id=b4ea621d5c894214a317b072289e756c) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:04.460471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:04.461939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c74dfa-e0d8-40df-ac98-54ec240ac227 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:04.467893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 562/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:04 2022] GET /identity/v3/role_assignments?user.id=b4ea621d5c894214a317b072289e756c&scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:06.602875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccf951-a66e-46cc-a9bd-d7928a24afad None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:06.603470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccf951-a66e-46cc-a9bd-d7928a24afad None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:06.603470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccf951-a66e-46cc-a9bd-d7928a24afad None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:06.604795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 556/1118] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:06.610601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:06.611025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:06.611117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:06.630788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:06.632036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:06.689648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:06.704752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 563/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:06.966812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccf951-a66e-46cc-a9bd-d7928a24afad None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:07.072128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccf951-a66e-46cc-a9bd-d7928a24afad None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:07.107007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:07.107645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:07.107921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:07.108187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] PATH_INFO: `/v3/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:07.109626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] RBAC: Authorizing `identity:get_role(role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:07.111009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:07.112373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 557/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:06 2022] GET /identity/v3/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:07.118577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:07.221066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e2ec-0e3e-4a15-83a4-b9089cdf61b8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:07.254805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:07.255474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:07.255756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:07.256014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] PATH_INFO: `/v3/users/b4ea621d5c894214a317b072289e756c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:07.257221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] RBAC: Authorizing `identity:get_user(user_id=b4ea621d5c894214a317b072289e756c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:07.258648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:07.260293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 564/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:07 2022] GET /identity/v3/users/b4ea621d5c894214a317b072289e756c => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:07.266451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:07.331000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:07.373870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3659c-8f39-4d45-a290-a383df5eda0d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:07.416876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:07.417517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:07.417792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:07.418062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:07.419260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:07.420722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a4d11-fa90-4a86-8f62-cdbb9c7e36e6 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:07.422393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 558/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:07 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:07.429140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:07.492727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:07.543557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb780b-dad7-4f7f-93b0-77ae4298a251 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:07.582731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:07.583421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:07.583688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:07.583957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71/users/b4ea621d5c894214a317b072289e756c/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:07.586325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] RBAC: Authorizing `identity:create_grant(project_id=f0687ad6e8534878bd9ff6ab9f15ac71, user_id=b4ea621d5c894214a317b072289e756c, role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:07.587974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873384-fa05-4b94-9a33-5adf836cb52c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:07.602380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 565/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:41:07 2022] PUT /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71/users/b4ea621d5c894214a317b072289e756c/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:09.733970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025e98-5ec1-4c9a-85aa-745325a4886a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:09.734535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025e98-5ec1-4c9a-85aa-745325a4886a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:09.734535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025e98-5ec1-4c9a-85aa-745325a4886a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:09.735903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 559/1124] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:09.741608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:09.742026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:09.742114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:09.761189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:09.762440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:09.832407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 566/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:10.099030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025e98-5ec1-4c9a-85aa-745325a4886a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:10.205324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025e98-5ec1-4c9a-85aa-745325a4886a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:10.236333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:10.243666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:10.244326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:10.244601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:10.244872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] PATH_INFO: `/v3/roles/14b1527e08be4a5195ffe47e1f684c22` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:10.246418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] RBAC: Authorizing `identity:get_role(role_id=14b1527e08be4a5195ffe47e1f684c22)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:10.247910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:10.249342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 560/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:10 2022] GET /identity/v3/roles/14b1527e08be4a5195ffe47e1f684c22 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:10.255814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:10.304183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:10.360146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc45746-2f4a-478e-b97a-ac7152f7c760 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:10.398184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:10.398827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:10.399105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:10.399406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] PATH_INFO: `/v3/users/b4ea621d5c894214a317b072289e756c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:10.400559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] RBAC: Authorizing `identity:get_user(user_id=b4ea621d5c894214a317b072289e756c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:10.402018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:10.403670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 567/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:10 2022] GET /identity/v3/users/b4ea621d5c894214a317b072289e756c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:10.409742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:10.526361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008886-bf88-4ffa-9cc5-21e93f6d3723 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:10.564074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:10.564681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:10.564956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:10.565220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:10.632207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:10.636002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:10.637803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:10.739570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 561/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:41:10.745803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:10.850495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d5002-5eb3-4250-97d4-dac89642793c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:10.866374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:10.888339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:10.888897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:10.889171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:10.889440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] PATH_INFO: `/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:10.890604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] RBAC: Authorizing `identity:get_project(project_id=f0687ad6e8534878bd9ff6ab9f15ac71)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:10.892103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6057300-6385-422a-bf82-61cb4bae1341 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:10.893812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 568/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:10 2022] GET /identity/v3/projects/f0687ad6e8534878bd9ff6ab9f15ac71 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:10.900750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:11.003019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:11.007251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b69ed-1f31-4210-8c4a-aa2829bd4a55 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:11.046805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:11.047451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:11.047723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:11.047978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:11.048736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] RBAC: Adding query filter params (role.id=14b1527e08be4a5195ffe47e1f684c22, scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71, user.id=b4ea621d5c894214a317b072289e756c) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:11.049389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:11.050918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb48905-f098-408f-8208-107e61a8d356 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:11.057110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 562/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:10 2022] GET /identity/v3/role_assignments?user.id=b4ea621d5c894214a317b072289e756c&scope.project.id=f0687ad6e8534878bd9ff6ab9f15ac71&role.id=14b1527e08be4a5195ffe47e1f684c22 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:13.276593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d344f-9f69-4303-ad76-2a0edee9499c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:13.277234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d344f-9f69-4303-ad76-2a0edee9499c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:13.277234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d344f-9f69-4303-ad76-2a0edee9499c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:13.278464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 569/1131] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:13.284221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:13.284789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:13.284789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:13.314446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:13.316180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:13.395039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 563/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:13.659612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d344f-9f69-4303-ad76-2a0edee9499c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:13.776938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d344f-9f69-4303-ad76-2a0edee9499c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:13.813644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:13.814280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:13.814562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:13.814833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:13.815998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:13.817450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458307-9df8-48a7-9ed5-03c4848a21e5 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:13.819048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 570/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:41:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:13.825974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:13.924893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:13.938558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a98661-f6d4-41a2-9434-65e08dfad9b9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:13.981579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:13.982197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:13.982466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:13.982737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:13.983412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:13.984863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784ed41-54db-4dff-aa98-7315931c27e2 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:14.017833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 564/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:41:13 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:41:16.229316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e5a3-2c68-4b1b-a17d-502c97b174d0 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:16.229888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e5a3-2c68-4b1b-a17d-502c97b174d0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:16.229888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612e5a3-2c68-4b1b-a17d-502c97b174d0 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:16.231200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 571/1135] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:16.238096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:16.238534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:16.238657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:16.261064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:16.262289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:16.334236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 565/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:16.602966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612e5a3-2c68-4b1b-a17d-502c97b174d0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:16.664697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:16.719971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612e5a3-2c68-4b1b-a17d-502c97b174d0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:16.758015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:16.758951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:16.759306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:16.759631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:16.761454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:16.763682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f97713-0858-4c86-bd34-5fd611fe1e2f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:16.766053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 572/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:16.773106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:16.854994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:16.882770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df51d6-47eb-4e42-ba15-76515dd98309 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:16.918943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:16.919583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:16.919822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:16.920117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:16.920809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:16.922265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211754c-7081-4166-bb1e-e503f3b54a41 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:16.946496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 566/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:41:16 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:41:19.163575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adda3b3-c14a-45f9-84f8-16e6ee78f1ed None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:19.164119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adda3b3-c14a-45f9-84f8-16e6ee78f1ed None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:19.164119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adda3b3-c14a-45f9-84f8-16e6ee78f1ed None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:19.165509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 573/1139] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:19.171063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:19.171466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:19.171640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:19.191054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:19.192278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:19.261984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 567/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:19.524137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adda3b3-c14a-45f9-84f8-16e6ee78f1ed None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:19.627965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adda3b3-c14a-45f9-84f8-16e6ee78f1ed None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:19.645485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:19.663174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:19.663842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:19.664118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:19.664384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:19.676106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:19.680925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:19.682049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:19.683131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 574/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:41:19.688364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:19.756048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:19.804326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8183737-eda4-4877-9572-1ced955630d7 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:19.839662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:19.840263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:19.840541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:19.840797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:19.841455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:19.841752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:19.843125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:19.851521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 568/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:19.857761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:19.965387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d8e03-830a-4a08-a7f9-a683c27a1391 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:19.998758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:19.999420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:19.999692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:19.999955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] PATH_INFO: `/v3/users/0d8b383177004c9099dc95fdc09c0d2b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:20.012921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] RBAC: Authorizing `identity:get_user(user_id=0d8b383177004c9099dc95fdc09c0d2b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:20.014350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:20.015968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 575/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:19 2022] GET /identity/v3/users/0d8b383177004c9099dc95fdc09c0d2b => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:20.022013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:20.126592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620649c-ccdf-4e67-b35f-064b0c95e09e None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:20.148969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:20.161541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:20.162183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:20.162435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:20.162702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:20.230436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:20.231926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:20.338427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 569/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:20.345187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:20.389087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:20.460435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c41a3a-3277-4e93-8f43-4167f669a96d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:20.496466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:20.497100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:20.497372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:20.497640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] PATH_INFO: `/v3/projects/0f5f4647281a48e48b016490cb0a59b2` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:20.498794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] RBAC: Authorizing `identity:get_project(project_id=0f5f4647281a48e48b016490cb0a59b2)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:20.500255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfe11a-90e9-4176-90b0-bf1470f3c88c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:20.501917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 576/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:20 2022] GET /identity/v3/projects/0f5f4647281a48e48b016490cb0a59b2 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:20.508464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:20.534080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:20.618206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a698-ce82-4db3-9402-91102d13d23b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:20.657402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:20.658008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:20.658263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:20.658546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:20.659345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=0f5f4647281a48e48b016490cb0a59b2, user.id=0d8b383177004c9099dc95fdc09c0d2b) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:20.659607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:20.661070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf514-5ca1-4849-a410-0aa59d6cb8d7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:20.666799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 570/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:20 2022] GET /identity/v3/role_assignments?user.id=0d8b383177004c9099dc95fdc09c0d2b&scope.project.id=0f5f4647281a48e48b016490cb0a59b2&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:22.782367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8901-f870-4a33-b5a5-b18282de00a9 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:22.782934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8901-f870-4a33-b5a5-b18282de00a9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:22.782934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8901-f870-4a33-b5a5-b18282de00a9 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:22.784302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 577/1147] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:22.789890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:22.790372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:22.790372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:22.809376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:22.810557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:22.879791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 571/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:41:23.154536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c8901-f870-4a33-b5a5-b18282de00a9 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:23.265448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c8901-f870-4a33-b5a5-b18282de00a9 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:23.267815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:23.303689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:23.304317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:23.304593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:23.304858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:23.315849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:23.320671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:23.321711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:23.322596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 578/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:23.328194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:23.346297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:23.431124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560c03d-3ef1-44aa-be3b-8155d6804ade None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:23.465020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:23.465627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:23.465905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:23.466171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:23.466842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:23.467141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:23.468558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:23.476701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 572/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:23.482975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:23.589264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23197e-8752-4dab-9778-59ae092e8bfd None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:23.623779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:23.624408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:23.624677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:23.624946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] PATH_INFO: `/v3/users/0d8b383177004c9099dc95fdc09c0d2b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:23.626088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] RBAC: Authorizing `identity:get_user(user_id=0d8b383177004c9099dc95fdc09c0d2b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:23.627602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:23.629192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 579/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:23 2022] GET /identity/v3/users/0d8b383177004c9099dc95fdc09c0d2b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:23.635589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:23.718147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:23.742918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d764-8925-4fbf-ab9d-630b706bf7d9 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:23.778442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:23.779069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:23.779367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:23.779637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] PATH_INFO: `/v3/projects/0f5f4647281a48e48b016490cb0a59b2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:23.780815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] RBAC: Authorizing `identity:get_project(project_id=0f5f4647281a48e48b016490cb0a59b2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:23.782272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a45746-b2bf-4c36-9e33-bca094902c54 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:23.784012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 573/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:23 2022] GET /identity/v3/projects/0f5f4647281a48e48b016490cb0a59b2 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:23.790815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:23.829386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:23.901459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7117c-db78-4357-af7d-c015af34069f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:23.936787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:23.937453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:23.937732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:23.938004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] PATH_INFO: `/v3/projects/0f5f4647281a48e48b016490cb0a59b2/users/0d8b383177004c9099dc95fdc09c0d2b/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:23.940067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] RBAC: Authorizing `identity:create_grant(project_id=0f5f4647281a48e48b016490cb0a59b2, user_id=0d8b383177004c9099dc95fdc09c0d2b, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:23.941522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1ee28-8aaa-4332-a119-9713554a16ef None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:23.956919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 580/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:41:23 2022] PUT /identity/v3/projects/0f5f4647281a48e48b016490cb0a59b2/users/0d8b383177004c9099dc95fdc09c0d2b/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:26.081833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b3a3-dbe0-4fbb-aa3d-0f8ad331a3bc None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:26.082318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b3a3-dbe0-4fbb-aa3d-0f8ad331a3bc None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:26.082318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b3a3-dbe0-4fbb-aa3d-0f8ad331a3bc None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:26.083747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 574/1154] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:26.089258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:26.089697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:26.089785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:26.109527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:26.110737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:26.182048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 581/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:26.448559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b3a3-dbe0-4fbb-aa3d-0f8ad331a3bc None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:26.557216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b3a3-dbe0-4fbb-aa3d-0f8ad331a3bc None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:26.592985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:26.593608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:26.593883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:26.594151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:26.604894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:26.609666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:26.610737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:41:26.611737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 575/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:26.617330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:26.643175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:26.725928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31abd86-e689-4b9e-b3c1-a619b43a7c6b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:26.761838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:26.762446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:26.762719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:26.762984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:26.763840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:26.764152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:26.765505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:26.773802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 582/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:26.780160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:26.790115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:26.888235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499105-d0d7-4893-94be-2237a98f17cc None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:26.925350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:26.925979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:26.926249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:26.926515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] PATH_INFO: `/v3/users/0d8b383177004c9099dc95fdc09c0d2b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:26.927694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] RBAC: Authorizing `identity:get_user(user_id=0d8b383177004c9099dc95fdc09c0d2b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:26.929128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:26.930700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 576/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:26 2022] GET /identity/v3/users/0d8b383177004c9099dc95fdc09c0d2b => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:26.937244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:27.027488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:27.044831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344345e-8ae9-401a-9664-7a71a6534b33 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:27.084764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:27.085391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:27.085666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:27.085934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:27.156404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:27.157850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:27.242257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:27.260261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 583/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:27.266655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:27.344354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:27.375498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078c40-8b60-42b5-bbd0-ba97bddbb323 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:27.413663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:27.414287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:27.414558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:27.414823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] PATH_INFO: `/v3/projects/0f5f4647281a48e48b016490cb0a59b2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:27.415989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] RBAC: Authorizing `identity:get_project(project_id=0f5f4647281a48e48b016490cb0a59b2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:27.417436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094adff-dde1-4fd2-905b-2e721999e943 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:27.419134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 577/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:27 2022] GET /identity/v3/projects/0f5f4647281a48e48b016490cb0a59b2 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:27.426231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:27.578278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef0461-bb28-4f0b-93e9-39caff293e9d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:27.635939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:27.636866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:27.637365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:27.637867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:27.639071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=0f5f4647281a48e48b016490cb0a59b2, user.id=0d8b383177004c9099dc95fdc09c0d2b) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:27.639662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:27.642235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943e3c-e819-4ea6-8567-b11e96d9d460 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:27.652578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 584/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:27 2022] GET /identity/v3/role_assignments?user.id=0d8b383177004c9099dc95fdc09c0d2b&scope.project.id=0f5f4647281a48e48b016490cb0a59b2&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:29.846250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01f738-59e9-4327-9ace-dbf83b7ccf4e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:29.846816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01f738-59e9-4327-9ace-dbf83b7ccf4e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:29.846816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01f738-59e9-4327-9ace-dbf83b7ccf4e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:29.848177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 578/1162] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:29.853667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:29.854132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:29.854226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:29.873415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:29.874641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:29.920396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:29.950129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 585/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:30.213499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01f738-59e9-4327-9ace-dbf83b7ccf4e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:30.321052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01f738-59e9-4327-9ace-dbf83b7ccf4e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:30.356116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:30.356716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:30.356990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:30.357256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:30.369895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:30.426093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:30.427586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:30.525281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 579/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:30.531507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:30.638340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f40734-8ff3-41fe-ac95-a1058ceb09a5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:30.673420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:30.674086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:30.674358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:30.674614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:30.681568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:30.683076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-10c43d07-b946-4f73-978a-7193887790cf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:30.695112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:30.696231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:30.697144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 586/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:41:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:30.702763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:30.753903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:30.817908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cb62-a90e-4dfc-bcd1-a92bd3956cab None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:30.855684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:30.856414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:30.856525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:30.856814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:30.857472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:30.857769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:30.859178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163ccd4-8b47-4181-ae46-f109e83ba98d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:30.867744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 580/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Sep 12 23:41:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:32.997152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828fff-0612-40b2-aac2-bf3e2cb3af60 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:32.997722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828fff-0612-40b2-aac2-bf3e2cb3af60 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:32.997722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828fff-0612-40b2-aac2-bf3e2cb3af60 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:32.999030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 587/1167] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:33.004758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceca25-c743-4ae5-882d-03f2842cfd4f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:33.005167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceca25-c743-4ae5-882d-03f2842cfd4f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:33.005290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceca25-c743-4ae5-882d-03f2842cfd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:33.025048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e3ceca25-c743-4ae5-882d-03f2842cfd4f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:33.026249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e3ceca25-c743-4ae5-882d-03f2842cfd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:33.096783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 581/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:33.360817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48828fff-0612-40b2-aac2-bf3e2cb3af60 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:33.411549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:33.477464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48828fff-0612-40b2-aac2-bf3e2cb3af60 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:33.517919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:33.518526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:33.518801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:33.519066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:33.519775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:33.521144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10865c-71cb-4a4c-b6c1-ab2e63b3517e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:33.541191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 588/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Sep 12 23:41:33 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:41:35.719548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c926f-7be8-49bc-89aa-75bd32df1726 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:35.720207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c926f-7be8-49bc-89aa-75bd32df1726 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:35.720207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c926f-7be8-49bc-89aa-75bd32df1726 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:35.721579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 582/1170] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:35.727997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:35.728592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:35.728592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:35.748974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:35.750207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:35.825691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 589/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:36.095238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c926f-7be8-49bc-89aa-75bd32df1726 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:36.134859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:36.203798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c926f-7be8-49bc-89aa-75bd32df1726 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:36.243524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:36.244117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:36.244425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:36.244691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] PATH_INFO: `/v3/domains/30cfac46028d4ad3842daf07bd91e666` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:36.256988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=30cfac46028d4ad3842daf07bd91e666)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:36.258444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1cfd-2259-4da5-913d-46ec1505a07c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:36.260112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 583/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Sep 12 23:41:36 2022] GET /identity/v3/domains/30cfac46028d4ad3842daf07bd91e666 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:36.269061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:36.301396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:36.376344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b51a8-2b40-40ab-9ae7-a4afc0115695 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:36.413966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:36.414571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:36.414854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:36.415092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:36.415801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:36.417232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585b19c-90f1-49dd-ae98-11402d3c17ed None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:36.445853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 590/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:41:36 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:41:38.622209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ce66-96c5-4842-80cb-84bdfde842f1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:38.622795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ce66-96c5-4842-80cb-84bdfde842f1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:38.622795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002ce66-96c5-4842-80cb-84bdfde842f1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:38.624182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 584/1174] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:38.630485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:38.631041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:38.631041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:38.651826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:38.653061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:38.723688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 591/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:38.995717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002ce66-96c5-4842-80cb-84bdfde842f1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:39.093403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:39.101269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002ce66-96c5-4842-80cb-84bdfde842f1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:39.143051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:39.143703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:39.143973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:39.144238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] PATH_INFO: `/v3/domains/30cfac46028d4ad3842daf07bd91e666` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:39.145412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=30cfac46028d4ad3842daf07bd91e666)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:39.146847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a443a-a1cd-4f26-9c3d-7650b181dcb7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:39.148458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 585/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Sep 12 23:41:38 2022] GET /identity/v3/domains/30cfac46028d4ad3842daf07bd91e666 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:39.156679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:39.169838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:39.273814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a8f4-6d4b-4269-8de6-a415ae6a4ff2 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:39.316396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:39.317011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:39.317274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:39.317540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:39.318184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:39.319663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07566dd-2b9a-4728-8831-fdac725e2a82 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:39.343742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 592/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:41:39 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:41:41.553234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab708b-e6c8-486b-9de4-613d2e90490c None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:41.553773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab708b-e6c8-486b-9de4-613d2e90490c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:41.553773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab708b-e6c8-486b-9de4-613d2e90490c None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:41.555093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 586/1178] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:41.560831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:41.561241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:41.561329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:41.581084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:41.582297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:41.645659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:41.655247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 593/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:41.922349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab708b-e6c8-486b-9de4-613d2e90490c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:42.029096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab708b-e6c8-486b-9de4-613d2e90490c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:42.064596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:42.065255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:42.065525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:42.065786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:42.076965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:42.081850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:42.082940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:41:42.084175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 587/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:42.089230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:42.190704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cd118-2c1b-470a-9ce7-4126738eb5ba None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:42.227864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:42.228472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:42.228756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:42.229017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:42.229684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:42.229986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:42.231391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:42.239857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 594/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:42.246302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:42.275490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:42.355034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a82a62-5387-48a8-a093-06033fce4db5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:42.391819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:42.392576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:42.392831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:42.393098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] PATH_INFO: `/v3/users/83784142b8f642dc98e9e9fabbc22315` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:42.409052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] RBAC: Authorizing `identity:get_user(user_id=83784142b8f642dc98e9e9fabbc22315)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:42.410525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:42.412232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 588/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:42 2022] GET /identity/v3/users/83784142b8f642dc98e9e9fabbc22315 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:42.418311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:42.510537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:42.562693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d526e1b-8c05-4a2c-8c27-f1606162818d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:42.606247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:42.606860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:42.607130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:42.607422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:42.677633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:42.679096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:42.738237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:42.780704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 595/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:42.790109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:42.879245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:42.900387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d522fd6-1573-4b76-99b9-d9a78620afc3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:42.936985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:42.937628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:42.937894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:42.938158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] PATH_INFO: `/v3/projects/f73eed66e17841c2a757d37ab85af42d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:42.939449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] RBAC: Authorizing `identity:get_project(project_id=f73eed66e17841c2a757d37ab85af42d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:42.940922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474c813-b807-4a10-933c-8aba09393596 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:42.942607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 589/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:42 2022] GET /identity/v3/projects/f73eed66e17841c2a757d37ab85af42d => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:42.949103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:43.058024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4654b-9ea8-4047-936c-62c1dd3060b3 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:43.097768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:43.098378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:43.098657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:43.098925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:43.099774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f73eed66e17841c2a757d37ab85af42d, user.id=83784142b8f642dc98e9e9fabbc22315) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:43.100191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:43.101676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fb628-4bb7-48b1-9988-b25d8cb66331 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:43.107483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 596/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:42 2022] GET /identity/v3/role_assignments?user.id=83784142b8f642dc98e9e9fabbc22315&scope.project.id=f73eed66e17841c2a757d37ab85af42d&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:45.236600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115d3b-64ad-45d2-8645-e22d726d2afe None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:45.237224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115d3b-64ad-45d2-8645-e22d726d2afe None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:45.237224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115d3b-64ad-45d2-8645-e22d726d2afe None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:45.238495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 590/1186] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:45.244155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:45.244613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:45.244717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:45.264673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:45.265889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:45.284205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:45.338050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 597/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:45.601274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c115d3b-64ad-45d2-8645-e22d726d2afe None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:45.712508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c115d3b-64ad-45d2-8645-e22d726d2afe None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:45.748623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:45.749252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:45.749522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:45.749787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:45.760554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:45.765389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:45.766480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:41:45.767614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 591/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:45.773500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:45.880651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32671e-2769-4b3a-ac58-01cedaa245f5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:45.916332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:45.916935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:45.917218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:45.917488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:45.918155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:45.918448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:45.919899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:45.928651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 598/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:45.934947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:45.948500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:46.038586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cfa09-fe96-40f8-952e-cd1f0ee22f31 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:46.075525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:46.076160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:46.076395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:46.076682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] PATH_INFO: `/v3/users/83784142b8f642dc98e9e9fabbc22315` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:46.077828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] RBAC: Authorizing `identity:get_user(user_id=83784142b8f642dc98e9e9fabbc22315)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:46.079276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:46.081098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 592/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:45 2022] GET /identity/v3/users/83784142b8f642dc98e9e9fabbc22315 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:46.087425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:46.105652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:46.195400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e54d2-c809-4877-aacc-7ea9c34c579e None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:46.230582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:46.231250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:46.231529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:46.231808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] PATH_INFO: `/v3/projects/f73eed66e17841c2a757d37ab85af42d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:46.232984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] RBAC: Authorizing `identity:get_project(project_id=f73eed66e17841c2a757d37ab85af42d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:46.234418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeead7-107b-43c4-8bb0-fc53edd43a9b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:46.236168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 599/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:46 2022] GET /identity/v3/projects/f73eed66e17841c2a757d37ab85af42d => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:46.242619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:46.330828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:46.352273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a94555-0604-4f09-b1d8-eda10b467e8d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:46.387366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:46.388027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:46.388297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:46.388544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] PATH_INFO: `/v3/projects/f73eed66e17841c2a757d37ab85af42d/users/83784142b8f642dc98e9e9fabbc22315/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:46.390557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] RBAC: Authorizing `identity:create_grant(project_id=f73eed66e17841c2a757d37ab85af42d, user_id=83784142b8f642dc98e9e9fabbc22315, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:46.392061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6897d5-aae8-45cb-9442-d6e20b7771cf None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:46.407630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 593/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:41:46 2022] PUT /identity/v3/projects/f73eed66e17841c2a757d37ab85af42d/users/83784142b8f642dc98e9e9fabbc22315/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:41:48.539544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ba95-7999-4371-94b8-56ecf563428e None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:48.540066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ba95-7999-4371-94b8-56ecf563428e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:48.540066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ba95-7999-4371-94b8-56ecf563428e None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:48.541345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 600/1193] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:41:48.546788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:48.547627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:48.547735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:48.568000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:41:48.569261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:41:48.642477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 594/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:41:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:41:48.912701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ba95-7999-4371-94b8-56ecf563428e None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.001768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:49.022181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ba95-7999-4371-94b8-56ecf563428e None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:49.058551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:49.059203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:49.059510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:49.059787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:49.072242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:41:49.077083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:41:49.078090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:41:49.078984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 601/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:41:49.084558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.156681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:49.198736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257f0a6-2d6b-4286-a211-491c631aff9c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:49.236945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:49.237548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:49.237824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:49.238091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:49.238755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:49.239055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:49.240473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:49.249530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 595/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:41:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:49.255881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.360751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2891-73ce-4640-bf96-dc1c174a0622 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:49.401510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:49.402143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:49.402416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:49.402685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] PATH_INFO: `/v3/users/83784142b8f642dc98e9e9fabbc22315` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:49.403887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] RBAC: Authorizing `identity:get_user(user_id=83784142b8f642dc98e9e9fabbc22315)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:49.405322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:49.406920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 602/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Sep 12 23:41:49 2022] GET /identity/v3/users/83784142b8f642dc98e9e9fabbc22315 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:49.413821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.525316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905b600-3667-4c9e-a7e1-e6c0720c2919 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:49.557392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:49.564206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:49.564824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:49.565092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:49.565340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:49.637599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:49.639049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:49.745570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 596/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:41:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:41:49.751892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.765836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:49.858324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f084f-8bbd-4ecc-8a97-7de744a58087 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:49.895000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:49.895652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:49.895925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:49.896199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] PATH_INFO: `/v3/projects/f73eed66e17841c2a757d37ab85af42d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:49.897327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] RBAC: Authorizing `identity:get_project(project_id=f73eed66e17841c2a757d37ab85af42d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:49.898776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc6420-8dfc-47b1-8ccf-093f4d96aa95 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:49.900447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 603/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Sep 12 23:41:49 2022] GET /identity/v3/projects/f73eed66e17841c2a757d37ab85af42d => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:41:49.907982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:41:49.939368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:41:50.014236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda77e-ad1e-47c3-8715-8708a4efe17b None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:41:50.050227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:41:50.050832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:41:50.051110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:41:50.051477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:41:50.052249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=f73eed66e17841c2a757d37ab85af42d, user.id=83784142b8f642dc98e9e9fabbc22315) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:41:50.052540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:41:50.053975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d410f7-098b-4bde-9688-4ce5db3f7597 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:41:50.064137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 597/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:41:49 2022] GET /identity/v3/role_assignments?user.id=83784142b8f642dc98e9e9fabbc22315&scope.project.id=f73eed66e17841c2a757d37ab85af42d&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:41:59.890220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=61 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:00.060920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=53 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:48.973892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073e5d-27c4-4e5b-96cb-9f26b280bfde None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:48.974771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073e5d-27c4-4e5b-96cb-9f26b280bfde None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:48.974771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d073e5d-27c4-4e5b-96cb-9f26b280bfde None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:48.975878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 604/1201] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:42:48.982658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:48.983333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:48.983655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:49.015852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:42:49.017305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:42:49.136532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 598/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:42:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:42:49.687362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d073e5d-27c4-4e5b-96cb-9f26b280bfde None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:49.890540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d073e5d-27c4-4e5b-96cb-9f26b280bfde None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:49.956852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:49.957867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:49.958374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:49.958838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:49.978594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:49.982666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:42:50.003505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:50.004777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:42:50.005778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:42:50.005778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:50.005778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:42:50.007987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 605/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:42:50.021213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:50.208989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43111e-ef10-4cb2-8f59-550344b7ae6e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:50.259453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:50.267363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:50.268231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:50.268596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:50.268985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:50.269701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:50.270010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:50.271623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125fda5-f8ea-4d35-b04c-8c3fa46b806d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:50.299288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 599/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:50.300997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:50.397087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:50.510236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54e9f-0b8f-415b-80b1-38e3f669eba1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:50.587079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:50.587939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:50.588416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:50.588791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:50.589717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:50.591678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e96b52-9e61-4a09-9a72-4326bff09bc1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:50.633774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 606/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Sep 12 23:42:50 2022] POST /identity/v3/users => generated 251 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:42:53.323290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7a8ca-87e4-404c-8fbc-5e8895b2859e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:53.323816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7a8ca-87e4-404c-8fbc-5e8895b2859e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:53.324061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7a8ca-87e4-404c-8fbc-5e8895b2859e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:53.325929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 600/1206] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:42:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:42:53.332997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:53.333656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:53.333806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:53.363184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:42:53.365485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:42:53.461781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 607/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:42:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:42:53.760333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7a8ca-87e4-404c-8fbc-5e8895b2859e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:53.894633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7a8ca-87e4-404c-8fbc-5e8895b2859e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:53.948383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:53.949361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:53.949628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:53.949995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:53.963482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:42:53.969205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:53.971748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:42:53.972664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 601/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:42:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:42:53.980379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:54.027763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:54.109768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fabf5-173c-44f0-aa8f-ab996faf5f67 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:54.150569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:54.151380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:54.153236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:54.153635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:54.154430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:54.154836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:54.156217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:54.165554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 608/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:42:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:42:54.172009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:54.255139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:54.335831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635d66a-5e37-4056-82e4-f70c121412ce None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:54.388387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:54.389505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:54.389997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:54.390495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:54.404089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:54.405685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:42:54.427640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:54.431412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:42:54.432513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:42:54.432513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:54.432513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:42:54.432513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 602/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:42:54.435404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:54.544060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:54.548665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aaaa1-0f8f-44cf-a913-9b9cd19db2b9 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:54.586461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:54.586980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:54.587265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:54.587540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:54.588180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:54.588478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:54.589854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:54.602956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 609/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:54.609007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:54.753738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa1984-e17b-4065-9134-2bd3594e347c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:54.755910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:54.800855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:54.801510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:54.801830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:54.802098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:54.811020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:54.812609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:42:54.827287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:54.828370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:42:54.829401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 603/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:42:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:42:54.835642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:54.966083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59b8f-b5d0-40ce-abdc-e5991da69e68 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:55.011962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60b097-4125-4a99-a868-1551622763df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:55.013178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60b097-4125-4a99-a868-1551622763df None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:55.013764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60b097-4125-4a99-a868-1551622763df None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:55.014417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60b097-4125-4a99-a868-1551622763df None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:55.015899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60b097-4125-4a99-a868-1551622763df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:55.016571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60b097-4125-4a99-a868-1551622763df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:55.019243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60b097-4125-4a99-a868-1551622763df None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:55.043986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 610/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:42:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:42:55.050567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:55.180680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3869b4-7b90-4c7b-ae20-555aa726bc00 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:55.222209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:55.222729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:55.223062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:55.223356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:55.274131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:55.325310 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:55.327475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:55.497728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 604/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:42:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 12 23:42:55.501053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60b097-4125-4a99-a868-1551622763df None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:55.527513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:55.634196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60b097-4125-4a99-a868-1551622763df None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:55.675263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:55.675263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:55.675263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:55.675263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:55.694227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:55.696073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:42:55.716595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:55.717784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:42:55.720342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:42:55.720342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:42:55.720342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:42:55.720342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 611/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:42:55.724142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:55.816187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:55.880002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76453495-eb80-42d5-9e77-1d62f728c5ac None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:55.937535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:55.938607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:55.939100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:55.939615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:55.940850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:55.941395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:55.944058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:55.972151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 605/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:55.980737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:56.101840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:56.149133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c80d9-6456-42bc-932d-970e81b63608 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:56.193842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:56.194495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:56.194773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:56.195040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:56.204167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:56.206455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:42:56.229762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:56.231246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:42:56.232376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 612/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:42:56.238459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:56.403204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c82972-045e-4490-8435-1a778f22b640 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:56.432941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:56.450105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:56.451320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:56.451789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:56.452216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:56.453246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:56.453684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:56.456006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21c94c-6229-42fa-ab62-7c32fd11ad3a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:56.473349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 606/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:56.479599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:56.628923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7862e-d204-4695-97b6-28c1e4c1aa91 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:56.665149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:56.678848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:56.679507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:56.679923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:56.680039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:56.680952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=734ed3e452894cbbbe292b78390dc2ce) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:56.681166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:56.682753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8de089-2851-44ed-9fe3-084fc86a7992 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:56.695107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 613/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:42:56 2022] GET /identity/v3/role_assignments?user.id=734ed3e452894cbbbe292b78390dc2ce&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:59.050225 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a932-4070-44d2-9dbd-3c49a4a54c28 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:59.050787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a932-4070-44d2-9dbd-3c49a4a54c28 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:59.050787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a932-4070-44d2-9dbd-3c49a4a54c28 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:59.052397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 607/1220] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:42:59.059018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:59.059699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:59.059699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:59.085619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:42:59.087352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:42:59.173660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 614/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:42:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:42:59.452990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a932-4070-44d2-9dbd-3c49a4a54c28 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:59.581831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a932-4070-44d2-9dbd-3c49a4a54c28 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:59.624277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:59.624908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:59.625241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:59.625487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:59.637184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:42:59.641857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:42:59.643660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:42:59.645040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 608/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:42:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:42:59.649305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:59.799760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84468a6b-d3a0-4b8b-8dea-9db505ca0c1d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:42:59.833269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:42:59.848670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:42:59.849781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:42:59.850263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:42:59.850725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:42:59.852449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] RBAC: Adding query filter params (name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:42:59.853268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:42:59.856650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:42:59.868730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 615/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:42:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:42:59.875612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:42:59.939940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:00.021931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362c7e-fc02-43c3-bd36-28ca7b7b6619 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:00.057744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:00.058412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:00.058574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:00.058840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:00.071300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:00.072932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:00.089878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:00.091418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:00.092570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:00.092570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:00.092570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:00.092570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 609/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:00.097652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:00.215176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af707-121e-41b1-a48a-1483d9fd9266 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:00.256961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:00.257587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:00.257929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:00.258214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:00.258869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:00.259173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:00.260706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:00.276244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 616/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:00.282532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:00.414856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:00.425189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712cd03-457e-4d5a-8e40-7537e72d53e7 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:00.469845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:00.470497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:00.470785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:00.471054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:00.484343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:00.486340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:43:00.503884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:00.505031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:00.506128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 610/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:00.512311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:00.585050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:00.664642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd37f6-8719-467f-8f19-343a7a2d0318 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:00.722747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:00.723317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:00.723633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:00.723904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:00.724744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:00.725168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:00.726979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:00.748637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 617/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:43:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:43:00.755528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:00.887233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189a74-7106-4cbe-9fb1-64296bee47e3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:00.924420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:00.925049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:00.925393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:00.925589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:00.937675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:00.939343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:00.958406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:00.960415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:00.961985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:00.961985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:00.961985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:00.967308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 611/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:00.967608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:01.101051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114d6f-ee1a-4683-bbff-d912e798cd0b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:01.105681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:01.150219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:01.151061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:01.151459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:01.151856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:01.152829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:01.153251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:01.155508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:01.172349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 618/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:01.179524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:01.192316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:01.327458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e9eb-7a83-4faf-9ba6-f5a2cd6db136 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:01.381115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:01.381995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:01.382470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:01.382818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:01.394324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:01.396291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:43:01.410281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:01.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:01.414187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:01.419970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 612/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:01.421027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:01.552228 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbcd0-1b65-4307-a9bc-7ca6e9f69424 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:01.591044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:01.591637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:01.591967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:01.592219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:01.592903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:01.593210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:01.594774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c620d40-73d9-4b1b-ac30-30929b6ce56b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:01.608465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 619/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:01.615195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:01.717298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:01.761257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a509968-ea74-41e6-acf3-82eb1893ec01 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:01.809112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:01.809982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:01.810258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:01.810555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/734ed3e452894cbbbe292b78390dc2ce/roles/634688ea68ba497f8067e6f5cdd49801` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:01.829886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=734ed3e452894cbbbe292b78390dc2ce, role_id=634688ea68ba497f8067e6f5cdd49801)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:01.831856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672bf7c-3a0b-4c4f-8dd5-5e9c58dc8d7d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:01.848754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 613/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:43:01 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/734ed3e452894cbbbe292b78390dc2ce/roles/634688ea68ba497f8067e6f5cdd49801 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:43:04.353734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f75bd-b9b7-4b87-b6a8-4267c567270a None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:04.354296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f75bd-b9b7-4b87-b6a8-4267c567270a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:04.354296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f75bd-b9b7-4b87-b6a8-4267c567270a None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:04.355778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 620/1233] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:04.361757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859966-3830-445c-a813-29f26ba76d2c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:04.362618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859966-3830-445c-a813-29f26ba76d2c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:04.362618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859966-3830-445c-a813-29f26ba76d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:04.383700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-08859966-3830-445c-a813-29f26ba76d2c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:04.384939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-08859966-3830-445c-a813-29f26ba76d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:04.485349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 614/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:04.782851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f75bd-b9b7-4b87-b6a8-4267c567270a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:04.821006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:04.926097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f75bd-b9b7-4b87-b6a8-4267c567270a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:04.972720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:04.973929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:04.974493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:04.975126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:04.994188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:43:05.002754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:05.006324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 12 23:43:05.009514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:05.013770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 621/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Sep 12 23:43:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:05.015612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859966-3830-445c-a813-29f26ba76d2c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:05.071322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:05.144090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859966-3830-445c-a813-29f26ba76d2c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:05.193235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:05.193879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:05.194190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:05.194459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:05.195187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:05.195580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:05.197101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:05.208547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 615/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Sep 12 23:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:43:05.215312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:05.332565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:05.354436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b4b66-b474-4e0c-bac7-3d02d178463d None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:05.408410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:05.409209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:05.409456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:05.409719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:05.423203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:05.425016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:05.442590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:05.447885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:05.449661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:05.449661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:05.449661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:05.449661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 622/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:05.450506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:05.595335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c22876-4981-4227-bf1e-e303fe4f5e23 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:05.606608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:05.643526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:05.644243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:05.644585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:05.644920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:05.645635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:05.646003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:05.647699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:05.663176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 616/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:43:05.668930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:05.806296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c89ad4-49fa-49f8-9294-52ca04368caf None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:05.856236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:05.857037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:05.858505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:05.858817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:05.862417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:05.868942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:05.870894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:43:05.886486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:05.887727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:05.888835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 623/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:43:05.893776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:06.025589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96370551-3f26-4f67-abf3-c5299a2c1fa3 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:06.079115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:06.079850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:06.080129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:06.080431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:06.081303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:06.081603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:06.083098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:06.104617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 617/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:43:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:06.112032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:06.252580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c168514-f8a2-45ba-9b2c-07a85186d304 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:06.304284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:06.304920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:06.305221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:06.305571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:06.389319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:06.398008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:06.400014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:06.556865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 624/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:43:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:43:06.564378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:06.613620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:06.720235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8b17-fb79-4bd8-8e7b-7976219c127a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:06.765235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:06.765924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:06.766218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:06.766528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:06.780823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:06.782587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:06.804486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:06.805738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:06.806778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:06.806778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:06.806778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:06.806778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 618/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:06.812370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:06.936888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:06.945114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb54a66-2a9f-433f-9412-2e711e0d2bfe None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:06.992325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:06.992815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:06.993093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:06.993371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:06.994118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:06.994410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:06.996013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:07.013603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 625/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:07.020183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:07.142393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:07.174849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb856-5be9-492b-8582-ec2969c5b545 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:07.232789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:07.234921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:07.235539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:07.236011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:07.247487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:07.249931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:43:07.271475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:07.272991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:07.274458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 619/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:07.280505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:07.417586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfaadc-f4d8-4bf3-b79b-0c3684ac688f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:07.465115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:07.465978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:07.466445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:07.466971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:07.468021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:07.468335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:07.470842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95311b13-6c92-4146-93d6-e1eb1c4791b1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:07.491091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 626/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:07.502618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:07.643021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3f023-78c5-4070-ad22-ebec671df7bd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:07.689741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:07.690757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:07.691136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:07.691564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:07.692416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] RBAC: Adding query filter params (role.id=634688ea68ba497f8067e6f5cdd49801, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=734ed3e452894cbbbe292b78390dc2ce) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:07.693038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:07.694364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c7ef6-348a-46f5-ab66-b020dbe0295c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:07.702842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 620/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:43:07 2022] GET /identity/v3/role_assignments?user.id=734ed3e452894cbbbe292b78390dc2ce&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=634688ea68ba497f8067e6f5cdd49801 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:10.216822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9ec1-e73c-4153-ba96-1e2edb81b838 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:10.217363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9ec1-e73c-4153-ba96-1e2edb81b838 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:10.217363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b9ec1-e73c-4153-ba96-1e2edb81b838 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:10.218733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 627/1247] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:10.225676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:10.226193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:10.226419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:10.234826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:10.250920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:10.253041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:10.339594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 621/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:10.644197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b9ec1-e73c-4153-ba96-1e2edb81b838 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:10.662342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:10.761565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b9ec1-e73c-4153-ba96-1e2edb81b838 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:10.802642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:10.803360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:10.803641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:10.803954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:10.814753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:43:10.819707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:10.820811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:10.821719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 628/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:10.827877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:10.956135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388ce2-f2cd-4c87-a664-f1c456dda8ea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:10.987739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:10.993058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:10.994024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:10.994468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:10.995042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:10.995797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:10.996105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:10.997604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:11.006787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 622/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:11.013062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:11.108783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:11.136238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f44fd-6ea0-4f06-bb4d-f8b2e7663128 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:11.173018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:11.173660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:11.173938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:11.174205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:11.185659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:11.187104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:11.201825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:11.202804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:11.203781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:11.203781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:11.203781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:11.203781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 629/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:11.209181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:11.314649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ecf3b-20d0-4593-8d44-5237d43dd1bf None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:11.353616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:11.354396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:11.354679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:11.354947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:11.355756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:11.356064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:11.357921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:11.377775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 623/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:11.379458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:11.498234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d258d4-d1f0-4727-b10b-d17eff805cc6 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:11.528442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:11.538411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:11.539058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:11.539362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:11.539632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:11.549352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:11.550825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:43:11.562921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:11.563940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:11.565356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 630/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:11.570123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:11.630984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:11.676763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47d09-7c0e-4949-80c3-721a7fe6fc73 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:11.717055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:11.717803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:11.718080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:11.718348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:11.719171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:11.719574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:11.721381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:11.741230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 624/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:43:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:11.747399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:11.862696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78114-c333-49c6-a1e2-5fde0399fdd4 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:11.899737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:11.900262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:11.900535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:11.900899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:11.975194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:11.991771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:11.993496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:12.119318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 631/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:43:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:43:12.120622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:12.232260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c4990-a106-4bd4-a9b8-ff5aafa6b2e5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:12.248617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:12.274397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:12.275059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:12.275367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:12.275646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:12.287433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:12.288988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:12.305849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:12.306923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:12.308047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:12.308047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:12.308047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:12.308047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 625/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:12.313399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:12.420851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:12.434351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fdd11-1868-406f-995c-0282807fbf7b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:12.472887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:12.473506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:12.473780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:12.474045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:12.474704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:12.475007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:12.476489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:12.489959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 632/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:12.495735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:12.612521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680f3d-58e6-4434-9d55-ddf643eb868c None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:12.652242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:12.652902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:12.653177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:12.653443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:12.661747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:12.663074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:43:12.676285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:12.677254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:12.678143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 626/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:12.683583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:12.804599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da62e5-94ae-4a99-98d4-85bbeabc5dd1 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:12.839639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:12.840252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:12.840536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:12.840805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:12.841507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:12.841799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:12.843257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ba0b-49e2-443c-a04a-85d83112aa59 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:12.859430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 633/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:12.866248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:12.894694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:12.979095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826cf67-3137-4e6f-a69a-6a5b0b260d39 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:13.015738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:13.016346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:13.016735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:13.016849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:13.017597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=734ed3e452894cbbbe292b78390dc2ce) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:13.017900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:13.019386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e862be-07a1-403f-93b9-bce6d49ab6ac None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:13.025026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 627/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:43:12 2022] GET /identity/v3/role_assignments?user.id=734ed3e452894cbbbe292b78390dc2ce&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:43:15.154122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e318bd-a2fc-426f-b77b-adbfa288b06b None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:15.154678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e318bd-a2fc-426f-b77b-adbfa288b06b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:15.154678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e318bd-a2fc-426f-b77b-adbfa288b06b None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:15.156045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 634/1261] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:15.161547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:15.161994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:15.162085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:15.181781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:15.182987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:15.251427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 628/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:15.514727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e318bd-a2fc-426f-b77b-adbfa288b06b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:15.526051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:15.622819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e318bd-a2fc-426f-b77b-adbfa288b06b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:15.658599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:15.659206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:15.659537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:15.659806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:15.671450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:43:15.676215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:15.677278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:15.678218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 635/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:15.683888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:15.704405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:15.801057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de75cb6-43d7-4056-927c-4c50b7656dce None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:15.838414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:15.839026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:15.839332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:15.839608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:15.840281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:15.840580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:15.842002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:15.850472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 629/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:15.856582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:15.926459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:15.961798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be2184-4871-40e4-a751-aced5ea79132 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:15.996890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:15.997520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:15.997823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:15.998100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.012562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.014171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:16.032586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:16.033682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:16.034837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:16.034837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:16.034837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:16.034837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 636/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:16.039913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.126917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:16.148668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507dd8-82b2-4b3a-8da5-80ae50a5d6bd None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:16.186462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:16.187278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:16.187385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:16.187611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.188249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:16.188551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.189925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.201878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 630/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:16.207920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.315948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f311c0-9ceb-47b9-b851-fd87974d7242 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:16.338788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:16.355786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:16.356433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:16.356709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:16.357126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] PATH_INFO: `/v3/users/placement` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.374853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.376353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:43:16.388968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:16.391176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:16.393528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 637/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:16.396461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.512975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9935f-a0a9-49fd-bb8c-ae32d9575e20 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:16.560658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:16.561257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:16.561501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:16.561774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.562510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:16.562808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.564264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.572934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:16.582233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 631/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:16.588492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.693596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ac33-2096-48ea-8fe4-c6b9ab58308b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:16.728089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:16.728725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:16.729004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:16.729275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.740162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.741592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:16.756062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:16.757151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:16.758054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:16.758054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:16.758054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:16.758054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 638/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:16.763290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.869666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a054-cdff-4300-b39a-9192e376a6e8 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:16.907109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:16.907757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:16.908035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:16.908301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:16.908949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:16.909253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:16.910644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:16.923116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 632/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:16.929897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:16.978665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:17.071338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74be0ba-ab87-4742-8975-158c96ab31cf None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:17.109108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:17.109743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:17.110022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:17.110296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:17.117445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:17.118893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:43:17.131188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:17.132182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:17.133220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 639/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:17.138550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:17.198421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:17.246436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ed7b9-9826-4738-a704-6fee15d09131 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:17.283378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:17.283987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:17.284262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:17.284532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:17.285224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:17.285530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:17.286995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3913ed-c61d-481f-893f-84050b44c28b None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:17.299989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 633/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:17.306248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:17.402405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:17.412438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff910-ea3b-42ee-ac72-2cb2f9304f48 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:17.448438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:17.449103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:17.449363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:17.449636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] PATH_INFO: `/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/734ed3e452894cbbbe292b78390dc2ce/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:17.451896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4177ed41ebc442889c73d90f9c38b9e4, user_id=734ed3e452894cbbbe292b78390dc2ce, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:17.453336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf329b-fc59-436b-bd1e-78fe09fd1bf1 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:17.467675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 640/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Sep 12 23:43:17 2022] PUT /identity/v3/projects/4177ed41ebc442889c73d90f9c38b9e4/users/734ed3e452894cbbbe292b78390dc2ce/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:43:19.633589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9c0-e2b4-4e87-a0fa-cccfe08b381e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:19.634196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9c0-e2b4-4e87-a0fa-cccfe08b381e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:19.634196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9c0-e2b4-4e87-a0fa-cccfe08b381e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:19.635506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 634/1274] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:19.640931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:19.641347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:19.641470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:19.660700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:19.661893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:19.744896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 641/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:20.006676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a9c0-e2b4-4e87-a0fa-cccfe08b381e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.138047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a9c0-e2b4-4e87-a0fa-cccfe08b381e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:20.159889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:20.181810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:20.182612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:20.182883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:20.183146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:20.194996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 12 23:43:20.199814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:20.200773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:20.201769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 635/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:20.207166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.293453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:20.313682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd992c31-176f-45ed-a249-85fa233c823b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:20.348208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:20.348803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:20.349071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:20.349339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:20.349994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:20.350294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:20.351694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:20.359962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 642/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:20.366101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.473142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4c6bb-79ba-4666-8762-c9fddea6194a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:20.508559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:20.509172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:20.509438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:20.509703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:20.519849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:20.521310 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:20.535917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:20.536890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:20.537972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:20.537972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:20.537972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:20.537972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 636/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:20.543038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.645858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4bbd2-ee49-4ff3-bb16-73a87b450e5f None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:20.680758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:20.681358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:20.681626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:20.681890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:20.682526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:20.682818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:20.684349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:20.698543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 643/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:20.704607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.736589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:20.810474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1313-30ed-4fc1-866f-5c22b05bec20 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:20.845921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76628660-4701-4f9d-afae-48feffad967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:20.846545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628660-4701-4f9d-afae-48feffad967d None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:20.846817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628660-4701-4f9d-afae-48feffad967d None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:20.847090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628660-4701-4f9d-afae-48feffad967d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:20.854915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76628660-4701-4f9d-afae-48feffad967d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:20.856402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76628660-4701-4f9d-afae-48feffad967d None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-76628660-4701-4f9d-afae-48feffad967d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 12 23:43:20.867678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:20.868746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:20.869840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 637/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 12 23:43:20.874977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:20.879007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:20.980090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704640b0-da1d-4fa7-b9a8-0fe04497fe0c None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:21.015209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:21.015837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:21.016112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:21.016375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:21.017094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:21.017375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:21.018809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:21.036774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 644/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Sep 12 23:43:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:21.042932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76628660-4701-4f9d-afae-48feffad967d None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:21.142765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:21.152918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76628660-4701-4f9d-afae-48feffad967d None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:21.187696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:21.188313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:21.188589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:21.188857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:21.256594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:21.257978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:21.345471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:21.357352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 638/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Sep 12 23:43:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 12 23:43:21.363496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:21.432049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:21.472295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1520c1-7f94-4cbb-ace0-2bbb485da366 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:21.509280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:21.509831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:21.510101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:21.510364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:21.520628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:21.522072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 12 23:43:21.536767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:21.537844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 12 23:43:21.538749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 12 23:43:21.538749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 12 23:43:21.538749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:21.538749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 645/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:21.544188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:21.659004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01bddc-4217-4328-9ea5-f60d3f0ddd20 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:21.697383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:21.697900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:21.698176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:21.698446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:21.699101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:21.699436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:21.700907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:21.718978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 639/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Sep 12 23:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:21.725148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:21.833903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327128-e5ee-454a-afcc-4dc068ab2369 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:21.847916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:21.870382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:21.871017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:21.871322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:21.871597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:21.878846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:21.880338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 12 23:43:21.892040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:21.893121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: ERROR keystone.server.flask.application Sep 12 23:43:21.894265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 646/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Sep 12 23:43:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:21.903016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:21.970182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:22.035017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389eaff-ebbb-4841-9855-17c0a873573a None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:22.074567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:22.075205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:22.075600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:22.075878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:22.076591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:22.076894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:22.078346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b54e1-4d37-41da-b67c-4b4deada1e23 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:22.091995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 640/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Sep 12 23:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:22.098218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:22.209146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53573af4-9891-40ee-97dc-16fe347084d8 None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:22.245064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:22.245670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:22.245949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:22.246220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:22.246950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] RBAC: Adding query filter params (role.id=c8bd8a06edea4fb7962ff7412be096ff, scope.project.id=4177ed41ebc442889c73d90f9c38b9e4, user.id=734ed3e452894cbbbe292b78390dc2ce) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:22.247283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:22.248790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4442e-a05c-4cac-94a5-e892d8b62592 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:22.254658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 647/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Sep 12 23:43:22 2022] GET /identity/v3/role_assignments?user.id=734ed3e452894cbbbe292b78390dc2ce&scope.project.id=4177ed41ebc442889c73d90f9c38b9e4&role.id=c8bd8a06edea4fb7962ff7412be096ff => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:24.413014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3edf-8635-4534-80ba-c905df0aa3ef None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:24.413561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3edf-8635-4534-80ba-c905df0aa3ef None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:24.413561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3edf-8635-4534-80ba-c905df0aa3ef None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:24.414808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 641/1288] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:24.420640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:24.421128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:24.421128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:24.440739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:24.441941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:24.474171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:24.515864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 648/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:24.793789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb3edf-8635-4534-80ba-c905df0aa3ef None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:24.919667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb3edf-8635-4534-80ba-c905df0aa3ef None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:24.938878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:24.966401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:24.967194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:24.967496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:24.967758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:24.968457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:24.970089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:24.981056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:24.982394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:24.983423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 642/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:43:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:24.989262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:25.096228 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475b257-f994-450e-8b67-31d8c2aa66a4 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:25.132250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:25.132845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:25.133117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:25.133382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:25.134012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] RBAC: Adding query filter params (name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:25.134301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:25.135722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b550f5-db60-479c-b632-994c02f82ddd None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:25.140660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 649/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Sep 12 23:43:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:25.146742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:25.247886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9aab-b133-4e38-8f72-a597f526e2a5 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:25.284514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:25.285120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:25.285400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:25.285674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] PATH_INFO: `/v3/services` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:25.286305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:25.286604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:25.288053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174209d-1694-4b85-a16b-f2e646df7a38 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:25.292760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 643/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Sep 12 23:43:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:27.431517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd648-a970-4be6-ad62-46f3f9ff5914 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:27.432027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd648-a970-4be6-ad62-46f3f9ff5914 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:27.432027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd648-a970-4be6-ad62-46f3f9ff5914 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:27.433219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 650/1293] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:27.439671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac623e-64b6-4e43-9ff3-c3701b47325b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:27.440155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac623e-64b6-4e43-9ff3-c3701b47325b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:27.440155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac623e-64b6-4e43-9ff3-c3701b47325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:27.459254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-88ac623e-64b6-4e43-9ff3-c3701b47325b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:27.460418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-88ac623e-64b6-4e43-9ff3-c3701b47325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:27.491481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:27.536374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 644/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 12 23:43:27.819153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd648-a970-4be6-ad62-46f3f9ff5914 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:27.851366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:27.930246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd648-a970-4be6-ad62-46f3f9ff5914 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:27.969525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:27.970144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:27.970419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:27.970687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:27.971379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:27.972758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc297-0e24-470b-8460-98dda2be77a7 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:27.992793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 651/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Sep 12 23:43:27 2022] POST /identity/v3/services => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:43:30.137241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0384a-ef00-4d4c-bdc6-35286edc3ef6 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:30.138074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0384a-ef00-4d4c-bdc6-35286edc3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:30.138197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0384a-ef00-4d4c-bdc6-35286edc3ef6 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:30.139555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 645/1296] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:30.144758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:30.145197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:30.145290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:30.165171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:30.166403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:30.241170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 652/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:30.501095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0384a-ef00-4d4c-bdc6-35286edc3ef6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:30.614675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0384a-ef00-4d4c-bdc6-35286edc3ef6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:30.651569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:30.652205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:30.652481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:30.652810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] PATH_INFO: `/v3/services/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:30.653447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:30.654826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:30.664176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:30.665336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:30.666199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 646/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:43:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:30.672034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:30.692281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:30.791074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de76ea-3c74-4243-aab4-47fd8dc69eb7 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:30.830449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:30.831087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:30.831397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:30.831650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:30.832282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] RBAC: Adding query filter params (name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:30.832585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:30.833970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:30.839606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 653/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Sep 12 23:43:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:30.846848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:30.880450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:30.962174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047171-6499-41e2-9329-e874dd8bb64f None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:31.001446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:31.002072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:31.002354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:31.002629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:31.003366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] RBAC: Adding query filter params (interface=public, service_id=d11120412a6a4e5c8adffaeb049920e0, region_id=RegionOne) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:31.003677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:31.005078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0669362-4b28-464c-bbd2-316ba306b609 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:31.011156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 647/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Sep 12 23:43:30 2022] GET /identity/v3/endpoints?service_id=d11120412a6a4e5c8adffaeb049920e0&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:31.017567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:31.106769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:31.128360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2c4f9-2a4b-458f-84f7-6a1e8f59ad7b None admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:31.166993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:31.167940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:31.167940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:31.168546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:31.169249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:31.170621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c3693-812e-4a0c-a4fb-4a4031de8645 None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:31.178684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 654/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Sep 12 23:43:31 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:43:33.365000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9e2c-84e3-4f70-85e1-2f9674c3a4d2 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:33.365573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9e2c-84e3-4f70-85e1-2f9674c3a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:33.365573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9e2c-84e3-4f70-85e1-2f9674c3a4d2 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:33.366910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 648/1302] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:43:33.372813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:33.373273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:33.373383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:33.393549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:43:33.394761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:43:33.471191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 655/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:43:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 12 23:43:33.733804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9e2c-84e3-4f70-85e1-2f9674c3a4d2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:33.864777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:33.872686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9e2c-84e3-4f70-85e1-2f9674c3a4d2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:33.909781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:33.910476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:33.910802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:33.911118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:33.911821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:33.913304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 12 23:43:33.923434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 12 23:43:33.925014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.get_or_create( Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application with Lock( Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self._enter() Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application return self.creator() Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application created_value = creator( Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: ERROR keystone.server.flask.application Sep 12 23:43:33.925859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 649/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Sep 12 23:43:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 12 23:43:33.931053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:34.012373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:34.037617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fe198-5b45-4792-a392-c444c0bfc422 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:34.078215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:34.079359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:34.080002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:34.080622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] PATH_INFO: `/v3/services` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:34.082003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:43:34.082307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:34.083735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730119c5-4e85-40be-9200-44706f8a87ef None admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:34.089613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 656/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Sep 12 23:43:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:43:34.097917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:43:34.206794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e81cd-c590-43e2-a0eb-b4fd2d3d5066 None admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:43:34.242107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:43:34.242717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:43:34.242995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:43:34.243266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:43:34.243973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:43:34.245356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48463e-a169-4618-a8b8-e41fb699dbd1 None admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:43:34.266957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 650/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:43:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:43:44.086700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=30 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:43:44.262134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:43.833774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be92c1-a988-481d-966d-599988d0467f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:43.834395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be92c1-a988-481d-966d-599988d0467f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:43.834395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be92c1-a988-481d-966d-599988d0467f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:43.835700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 657/1307] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:43.841260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1029991-7e5c-4fdb-ba63-05833cdf9f1f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:43.841878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1029991-7e5c-4fdb-ba63-05833cdf9f1f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:43.842250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1029991-7e5c-4fdb-ba63-05833cdf9f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:43.929380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d1029991-7e5c-4fdb-ba63-05833cdf9f1f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:43.930607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d1029991-7e5c-4fdb-ba63-05833cdf9f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:44.085540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 651/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:44:44.379135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:44.379517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:44.379832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:44.381984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 658/1309] 10.222.0.26 () {54 vars in 965 bytes} [Mon Sep 12 23:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:44.388205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4bdc-bf16-4059-830b-8ccb4ba53248 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:44.388547 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4bdc-bf16-4059-830b-8ccb4ba53248 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:44.388736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4bdc-bf16-4059-830b-8ccb4ba53248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:44.478892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-42be4bdc-bf16-4059-830b-8ccb4ba53248 None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:44.480030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-42be4bdc-bf16-4059-830b-8ccb4ba53248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:44.503357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:44.568892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 652/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Mon Sep 12 23:44:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:44:44.577882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:44:44.579651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:44.715905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd8c33-5720-4a49-b36a-4d5034295c3c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:44:44.810052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:44:44.810682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:44.810965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:44.811258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:44.812912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:44.879879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:44.945624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:44:44.947129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e2f56-8571-48ee-89df-8fcd865b78ef service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:44:44.967153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 659/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:44:47.463729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:47.464290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:47.464290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:47.465563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 653/1312] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:47.471073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec13dea-1bd3-45b4-b831-82f6953bd306 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:47.471549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec13dea-1bd3-45b4-b831-82f6953bd306 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:47.471654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec13dea-1bd3-45b4-b831-82f6953bd306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:47.562798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7ec13dea-1bd3-45b4-b831-82f6953bd306 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:47.564060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7ec13dea-1bd3-45b4-b831-82f6953bd306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:47.669536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 660/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:44:47.956915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:44:47.958508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:48.067479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbe054-a73b-4d48-bd60-00786eb480c9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:44:48.141593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:48.144184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:44:48.144828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:48.145115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:48.145389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:48.147570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:48.273522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:44:48.275018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf539d5-fc6b-44be-962d-2843028520e4 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:44:48.294363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 654/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:44:50.801063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:50.801692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:50.801692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:50.803772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 661/1315] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:50.809793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43074-d42f-4749-99aa-9ca3ca146dcf None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:50.810331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43074-d42f-4749-99aa-9ca3ca146dcf None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:50.810331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc43074-d42f-4749-99aa-9ca3ca146dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:50.880876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9cc43074-d42f-4749-99aa-9ca3ca146dcf None None] MFA Rules not processed for user `1e961cef8d2643c480253e713a38bdde`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:50.882089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9cc43074-d42f-4749-99aa-9ca3ca146dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:50.938385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:50.986956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 655/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:44:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:44:51.273722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:44:51.275341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:51.285366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:51.389398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40eb63-3298-4af3-a648-559f56039f41 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:44:51.481532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:44:51.482587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:51.483107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:51.483680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:51.485429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:51.564964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:51.615082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:44:51.616555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430e0d5-3ca4-4351-a949-12b16dfe4be1 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:44:51.634801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 662/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:44:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:44:54.394894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:54.395473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:54.395473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:54.396746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 656/1318] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:54.402763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aebad-85c6-45e8-b281-0115cde8473e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:54.403310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aebad-85c6-45e8-b281-0115cde8473e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:54.403310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aebad-85c6-45e8-b281-0115cde8473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:54.477410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-340aebad-85c6-45e8-b281-0115cde8473e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:54.478799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-340aebad-85c6-45e8-b281-0115cde8473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:54.589522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 663/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:44:54.876461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:44:54.878374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:54.992386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ca6e-691e-426d-be2a-ca20e6b94de5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:44:55.071158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:44:55.071926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:55.072202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:55.072457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:55.074075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:55.081426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:55.192489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:44:55.193862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128bc6e-91b6-498b-bdb2-cabccb9fa0f5 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:44:55.213485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 657/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:44:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:44:58.068586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62bfc-d74e-44e4-a8df-534e00550571 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:58.069146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62bfc-d74e-44e4-a8df-534e00550571 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:58.069146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62bfc-d74e-44e4-a8df-534e00550571 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:58.070675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 664/1321] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:58.076124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75153d26-6be6-490f-a8af-543ff6dcf8ff None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:58.076546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75153d26-6be6-490f-a8af-543ff6dcf8ff None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:58.076636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75153d26-6be6-490f-a8af-543ff6dcf8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:58.148996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-75153d26-6be6-490f-a8af-543ff6dcf8ff None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:58.150625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-75153d26-6be6-490f-a8af-543ff6dcf8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:58.212100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:58.262052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 658/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:44:58.562296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:58.562581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:58.562840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:58.564278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 665/1323] 10.222.0.26 () {54 vars in 965 bytes} [Mon Sep 12 23:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:44:58.570820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab374-4ecc-4c75-bcb5-8a25f664b298 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:58.571211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab374-4ecc-4c75-bcb5-8a25f664b298 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:58.571642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eab374-4ecc-4c75-bcb5-8a25f664b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:58.661190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-13eab374-4ecc-4c75-bcb5-8a25f664b298 None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:44:58.662317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-13eab374-4ecc-4c75-bcb5-8a25f664b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:58.747175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 659/1324] 10.222.0.26 () {58 vars in 1062 bytes} [Mon Sep 12 23:44:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:44:58.756606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:44:58.758177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:58.780975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:58.877076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a46ff-5449-4e20-855b-cfdac441aae1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:44:58.961726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:44:58.962354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:44:58.962624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:44:58.962887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:44:58.964521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:44:59.042880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:44:59.085528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:44:59.087310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5534e8-97a1-47a6-93ee-7e11e4f22535 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:44:59.106837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 666/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:01.568857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:01.569453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:01.569453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:01.570793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 660/1326] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:01.576962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eea1f-6252-4f14-836c-0ea77da397c1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:01.577474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eea1f-6252-4f14-836c-0ea77da397c1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:01.577586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eea1f-6252-4f14-836c-0ea77da397c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:01.652887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-353eea1f-6252-4f14-836c-0ea77da397c1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:01.654101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-353eea1f-6252-4f14-836c-0ea77da397c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:01.764998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 667/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:02.053855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:02.055682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:02.139486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:02.196233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec043be-d253-4df6-91f5-452191fadd5f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:02.281925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:02.282611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:02.282879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:02.283167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:02.284890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:02.424605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:02.427465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:02.429361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9eab6-de0a-4a03-bc26-5e5c31483cef service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:02.449155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 661/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:04.704807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:04.705416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:04.705416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:04.707471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 668/1329] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:04.713040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468400e-4ba4-4ff5-b023-4955eabc0450 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:04.713588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468400e-4ba4-4ff5-b023-4955eabc0450 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:04.713588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468400e-4ba4-4ff5-b023-4955eabc0450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:04.789203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7468400e-4ba4-4ff5-b023-4955eabc0450 None None] MFA Rules not processed for user `1e961cef8d2643c480253e713a38bdde`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:04.790898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7468400e-4ba4-4ff5-b023-4955eabc0450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:04.898091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 662/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:05.186562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:05.188170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:05.224472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:05.323094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c51cb-e06e-4b0d-9255-f7f69606396e None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:05.403968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:05.404588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:05.404866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:05.405144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:05.406748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:05.495410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:05.533510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:05.534986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4210045-3218-4917-9f89-774d8aa7d10a service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:05.557913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 669/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:08.182309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:08.182309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:08.182309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:08.183664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 663/1332] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:08.189068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557461a3-412f-4cf1-be75-6539d5b572f1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:08.189529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557461a3-412f-4cf1-be75-6539d5b572f1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:08.189633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557461a3-412f-4cf1-be75-6539d5b572f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:08.266001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-557461a3-412f-4cf1-be75-6539d5b572f1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:08.267260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-557461a3-412f-4cf1-be75-6539d5b572f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:08.374558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 670/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:08.661995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:08.663583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:08.743016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:08.781952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8df944-577f-49f5-95bc-540920eed35e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:08.866147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:08.866727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:08.867026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:08.867286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:08.869021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:08.988377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:08.989838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ee78-d59b-4b42-b973-4a6abc0d3797 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:09.000723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:09.011758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 664/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:11.233621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:11.234319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:11.234319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:11.235576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 671/1335] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:11.240998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307692c-972e-49a2-9cdb-4adc22009b44 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:11.241450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307692c-972e-49a2-9cdb-4adc22009b44 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:11.241553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307692c-972e-49a2-9cdb-4adc22009b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:11.311822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0307692c-972e-49a2-9cdb-4adc22009b44 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:11.313070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0307692c-972e-49a2-9cdb-4adc22009b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:11.412515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 665/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:11.693858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:11.695826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:11.739380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:11.806140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687734e-5a3e-4681-9c0e-85fb481c454c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:11.881124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:11.881734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:11.882011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:11.882290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:11.884058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:11.974956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:12.002309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:12.003803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557664e-9e57-4b7e-a00f-9771db7d2653 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:12.022930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 672/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:14.578957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:14.579649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:14.579649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:14.580965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 666/1338] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:14.587214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89943-45fa-4092-9248-5c829e2f5761 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:14.587832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89943-45fa-4092-9248-5c829e2f5761 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:14.587832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e89943-45fa-4092-9248-5c829e2f5761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:14.665145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c2e89943-45fa-4092-9248-5c829e2f5761 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:14.666390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c2e89943-45fa-4092-9248-5c829e2f5761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:14.772928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 673/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:15.052462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:15.054318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:15.138921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:15.168783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826672c7-7d54-4793-9aff-c04bd00cd1fd None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:15.250729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:15.251379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:15.251652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:15.251918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:15.253534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:15.384899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:15.386387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f7f97-3fcb-4543-8e45-dfb2a2c9d314 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:15.406009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 667/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:19.912368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:19.912941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:19.912941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:19.914585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 674/1341] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:19.920900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214cc40-7e0f-4a36-87f6-051474f241ff None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:19.921489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214cc40-7e0f-4a36-87f6-051474f241ff None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:19.921489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214cc40-7e0f-4a36-87f6-051474f241ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:19.927870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:20.002730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5214cc40-7e0f-4a36-87f6-051474f241ff None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:20.003887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5214cc40-7e0f-4a36-87f6-051474f241ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:20.112277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 668/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:20.396809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:20.398435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:20.466673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:20.559568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f64be-59b1-4496-8a60-bddd4166ba42 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:20.648598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:20.649082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:20.649430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:20.649698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:20.651374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:20.762771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:20.787336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:20.790245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bebba-410f-4fca-baec-7ffc8202ca82 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:20.820863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 675/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:23.845201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:23.845819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:23.845819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:23.847463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 669/1344] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:23.853850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e536f-2f2d-4b79-a2d7-19150bc35ad5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:23.854615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e536f-2f2d-4b79-a2d7-19150bc35ad5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:23.854615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e536f-2f2d-4b79-a2d7-19150bc35ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:23.930795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-206e536f-2f2d-4b79-a2d7-19150bc35ad5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:23.932059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-206e536f-2f2d-4b79-a2d7-19150bc35ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:24.040150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 676/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:24.319454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:24.321086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:24.420155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:24.438653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09d171-0eb3-42e9-8061-5f272b3f0bd3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:24.561671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:24.562343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:24.562612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:24.562876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:24.564663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:24.707779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:24.709620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01669ad8-6e24-4c21-ae23-4433e3092598 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:24.738221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 670/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:29.401579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:29.402119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:29.402119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:29.403852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 677/1347] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:29.411082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48066a8-c3ed-4325-9a49-ead2103759f3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:29.411743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48066a8-c3ed-4325-9a49-ead2103759f3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:29.411743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48066a8-c3ed-4325-9a49-ead2103759f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:29.436073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:29.495258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d48066a8-c3ed-4325-9a49-ead2103759f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:29.496517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d48066a8-c3ed-4325-9a49-ead2103759f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:29.611879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 671/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:29.892486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:29.894128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:29.958680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:30.011642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1ab6-9e76-4fc2-9889-bcef3e6b0f8c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:30.096875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:30.097497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:30.097770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:30.098039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:30.099648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:30.213708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:30.217571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:30.219524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1d9a-e253-4c94-b066-690c0d439ee6 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:30.238370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 678/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:35.650349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbcc8-3ac2-4345-ab85-70eefdccc26e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:35.651125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbcc8-3ac2-4345-ab85-70eefdccc26e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:35.651611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefbcc8-3ac2-4345-ab85-70eefdccc26e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:35.656668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 672/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:45:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:35.664873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80277c-e32e-4298-99a6-2b833e9fb012 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:35.665421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80277c-e32e-4298-99a6-2b833e9fb012 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:35.665747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80277c-e32e-4298-99a6-2b833e9fb012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:35.802302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2a80277c-e32e-4298-99a6-2b833e9fb012 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:35.804846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2a80277c-e32e-4298-99a6-2b833e9fb012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:35.992664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 679/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:45:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:39.611772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=59 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:41.438362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3e90-91c9-42b4-94fe-e8706359777a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:41.438997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3e90-91c9-42b4-94fe-e8706359777a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:41.438997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3e90-91c9-42b4-94fe-e8706359777a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:41.440468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 673/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:41.445840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cd39-21e9-4d45-8205-f143fe8cc324 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:41.446851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cd39-21e9-4d45-8205-f143fe8cc324 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:41.446851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cd39-21e9-4d45-8205-f143fe8cc324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:41.514399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:41.552609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3df3cd39-21e9-4d45-8205-f143fe8cc324 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:41.554056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3df3cd39-21e9-4d45-8205-f143fe8cc324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:41.728094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 680/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:45:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:42.974113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2c01c-03a7-4a5f-a0e9-2be6b6a9255b None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:42.974971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2c01c-03a7-4a5f-a0e9-2be6b6a9255b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:42.974971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2c01c-03a7-4a5f-a0e9-2be6b6a9255b None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:42.976902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 674/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:42.982757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355fb4d-9a6a-4181-a372-321027957b70 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:42.983375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355fb4d-9a6a-4181-a372-321027957b70 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:42.983503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355fb4d-9a6a-4181-a372-321027957b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:43.083991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3355fb4d-9a6a-4181-a372-321027957b70 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:43.085300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3355fb4d-9a6a-4181-a372-321027957b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:43.140669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:43.232628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 681/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:45:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:46.833399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ae00-0017-4600-a373-3fa86de706e6 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:46.834012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ae00-0017-4600-a373-3fa86de706e6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:46.834012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ae00-0017-4600-a373-3fa86de706e6 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:46.835309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 675/1356] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:45:46.841913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b2724-efe3-4a09-98a6-288b16326cc7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:46.842431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b2724-efe3-4a09-98a6-288b16326cc7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:46.842983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b2724-efe3-4a09-98a6-288b16326cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:46.977466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2e5b2724-efe3-4a09-98a6-288b16326cc7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:46.981138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2e5b2724-efe3-4a09-98a6-288b16326cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:47.120604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 682/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:47.424484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:47.425057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:47.425495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:47.427592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 676/1358] 10.222.0.26 () {54 vars in 962 bytes} [Mon Sep 12 23:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:47.436031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58a486-41e3-4b60-93aa-9012cef4cd2f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:47.436307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58a486-41e3-4b60-93aa-9012cef4cd2f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:47.436579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58a486-41e3-4b60-93aa-9012cef4cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:47.460258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:47.540697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3c58a486-41e3-4b60-93aa-9012cef4cd2f None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:47.541878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3c58a486-41e3-4b60-93aa-9012cef4cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:47.691168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 683/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Sep 12 23:45:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:47.699239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:47.700810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:47.891970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de1375-6951-4b5f-8ec9-84a5fd21499c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:47.978477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:48.045434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:48.046104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:48.046407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:48.046879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:48.049335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:48.180017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:48.181888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b61987-c640-43ad-8b5a-c81026e2c043 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:48.216885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 677/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:51.045882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:51.046683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:51.046683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:51.047855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 684/1361] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:45:51.054087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e23391-985b-45b3-afc2-59588c9096d9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:51.054904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e23391-985b-45b3-afc2-59588c9096d9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:51.054904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e23391-985b-45b3-afc2-59588c9096d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:51.134902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-89e23391-985b-45b3-afc2-59588c9096d9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:51.137130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-89e23391-985b-45b3-afc2-59588c9096d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:51.207782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:51.306273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 678/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:51.600385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:51.602141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:51.700184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:51.814820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905ec79-1746-4e18-96fd-996478529e71 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:51.965106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:51.965735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:51.966021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:51.966283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:51.968035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:51.995634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:52.099761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:52.101386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460fbfe-e94a-4234-9bfa-eedd83698f72 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:52.123013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 685/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:54.739174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e58d-ff66-4423-8439-0c6ca3faa16e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:54.739762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e58d-ff66-4423-8439-0c6ca3faa16e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:54.739762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e58d-ff66-4423-8439-0c6ca3faa16e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:54.743277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 679/1364] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:54.748658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9fd8-6c9b-4681-9ad6-df8baee8e823 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:54.749523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9fd8-6c9b-4681-9ad6-df8baee8e823 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:54.749523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9fd8-6c9b-4681-9ad6-df8baee8e823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:54.849775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f4ac9fd8-6c9b-4681-9ad6-df8baee8e823 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:54.851698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f4ac9fd8-6c9b-4681-9ad6-df8baee8e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:54.976077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:54.987878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 686/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:55.323353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:55.323834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:55.324166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:55.326011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 680/1366] 10.222.0.26 () {54 vars in 962 bytes} [Mon Sep 12 23:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:55.334422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070505-50c6-4dd9-ac23-fe4da43a73f4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:55.335015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070505-50c6-4dd9-ac23-fe4da43a73f4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:55.335670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070505-50c6-4dd9-ac23-fe4da43a73f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:55.441024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-93070505-50c6-4dd9-ac23-fe4da43a73f4 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:55.442416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-93070505-50c6-4dd9-ac23-fe4da43a73f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:55.620418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 687/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Sep 12 23:45:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:55.633298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:55.635981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:55.862375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73bbd4-e654-4d4f-8669-f154311bc769 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:55.893960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:56.021611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21312bab-7856-4d6a-b9ff-ee0e038d1e6e None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:56.022202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21312bab-7856-4d6a-b9ff-ee0e038d1e6e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:56.022202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21312bab-7856-4d6a-b9ff-ee0e038d1e6e None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:56.023710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 688/1368] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:56.029366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c229d-e5d4-46f4-bb91-c955998180c3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:56.029609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c229d-e5d4-46f4-bb91-c955998180c3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:56.029883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c229d-e5d4-46f4-bb91-c955998180c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:56.043015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:56.044004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:56.044401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:56.044770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:56.046878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:56.129861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:56.141862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b30c229d-e5d4-46f4-bb91-c955998180c3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:56.144079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b30c229d-e5d4-46f4-bb91-c955998180c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:56.176913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:56.180117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a6e12-d9df-4fff-928f-40129f0b9131 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:56.211628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 681/1369] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:56.264341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 689/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:45:56.621485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:56.621485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:56.621485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:56.625724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 682/1371] 10.222.0.26 () {54 vars in 964 bytes} [Mon Sep 12 23:45:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:45:56.636525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761538e-b087-4d28-b995-0deeb496b74e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:56.636525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761538e-b087-4d28-b995-0deeb496b74e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:56.636525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761538e-b087-4d28-b995-0deeb496b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:56.729701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a761538e-b087-4d28-b995-0deeb496b74e None None] MFA Rules not processed for user `65d52c53e1244d25b0e16374101ea139`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:56.730933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a761538e-b087-4d28-b995-0deeb496b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:56.811812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:56.819279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 690/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Sep 12 23:45:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:56.827265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:56.829642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:56.861132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:56.964785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e83f1-84df-45ad-a76c-6f544bbc579e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:57.090481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:57.091517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:57.091972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:57.092417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:57.095605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:57.212586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:57.296633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:57.301341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce2fb4-542f-42b0-a893-9487554b5904 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:57.344041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 683/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:45:58.563759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:58.564626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:58.564815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:58.566471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 691/1374] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:45:58.572424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c38b9-6062-48c6-84f5-6741df88a8ea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:58.573080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c38b9-6062-48c6-84f5-6741df88a8ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:58.573080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c38b9-6062-48c6-84f5-6741df88a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:58.653837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f92c38b9-6062-48c6-84f5-6741df88a8ea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:45:58.655068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f92c38b9-6062-48c6-84f5-6741df88a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:58.775588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 684/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:45:59.063900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:45:59.065553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:59.279736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad707a-c69f-47fd-98fc-f86b4b090d6a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:45:59.380958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:45:59.450151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:45:59.450797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:45:59.451075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:45:59.451389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:45:59.453099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:45:59.564847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:45:59.568087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b510-b194-4e4e-b8eb-e86885a9da80 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:45:59.603842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 692/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:02.010648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:02.011597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:02.011597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:02.012784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 685/1377] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:46:02.018592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5566-72da-41c7-a0a9-e60c77cf1fee None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:02.019320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5566-72da-41c7-a0a9-e60c77cf1fee None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:02.019320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5566-72da-41c7-a0a9-e60c77cf1fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:02.150704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a82e5566-72da-41c7-a0a9-e60c77cf1fee None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:02.152223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a82e5566-72da-41c7-a0a9-e60c77cf1fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:02.202553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:02.293588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 693/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:02.588631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:02.590565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:02.637708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:02.834282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99beaf3-ed3a-45f9-856e-c5c8797ed03c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:03.073023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:03.074197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:03.075082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:03.076024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:03.079744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:03.081707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:03.240363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:03.242135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046dec3-c50d-4f78-91b8-31550848800e service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:03.264715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 686/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:05.751894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:05.752424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:05.753000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:05.755856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 694/1380] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:05.762842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4df691-9313-4865-b0f5-471f4b908399 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:05.763551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4df691-9313-4865-b0f5-471f4b908399 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:05.763551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4df691-9313-4865-b0f5-471f4b908399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:05.868641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-af4df691-9313-4865-b0f5-471f4b908399 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:05.870279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-af4df691-9313-4865-b0f5-471f4b908399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:05.951988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:05.988885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 687/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:46:06.319097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:06.321331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:06.503399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e9f98-5c12-411b-85ab-a88e3add8b7b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:06.523546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:06.683561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:06.684346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:06.684706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:06.684972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:06.686840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:06.799735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:06.801629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613d9eb-24aa-4d0f-8618-35fb9269224f service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:06.827312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 695/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:09.163078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:09.163723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:09.163723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:09.165818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 688/1383] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:09.172174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958762-0925-4b23-8fcb-c3e33d180e03 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:09.172785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958762-0925-4b23-8fcb-c3e33d180e03 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:09.172785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63958762-0925-4b23-8fcb-c3e33d180e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:09.199116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:09.259173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-63958762-0925-4b23-8fcb-c3e33d180e03 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:09.260602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-63958762-0925-4b23-8fcb-c3e33d180e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:09.387119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 696/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:09.678242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:09.679896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:09.902199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1dff9-f078-47bb-9786-82e12daa1275 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:10.001398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:10.096408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:10.097054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:10.097327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:10.097604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:10.099345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:10.218340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:10.220018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45f770-5862-454c-8da0-dcea2ad0a2a3 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:10.240805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 689/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:12.888983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:12.889570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:12.889956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:12.891632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 697/1386] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:12.897702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30412-127b-43c4-85cc-77468e02b051 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:12.898248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30412-127b-43c4-85cc-77468e02b051 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:12.898248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30412-127b-43c4-85cc-77468e02b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:12.977157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e8a30412-127b-43c4-85cc-77468e02b051 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:12.978453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e8a30412-127b-43c4-85cc-77468e02b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:12.987828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:13.093570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 690/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:46:13.387186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:13.390073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:13.515781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:13.580206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752f57-2a04-488d-a33d-6b022a2cb922 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:13.752282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:13.753391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:13.753939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:13.754379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:13.756793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:13.875275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:13.918752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:13.920730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510b58d-ff47-469f-b5b7-cbd0b2917379 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:13.948053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 698/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:14.586833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d992f-a696-4fd7-babc-6ffb0fd41e98 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:14.587608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d992f-a696-4fd7-babc-6ffb0fd41e98 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:14.587608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d992f-a696-4fd7-babc-6ffb0fd41e98 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:14.589168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 691/1389] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:46:14.595193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332c82-acfd-41d2-9f18-fce7cc2476cc None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:14.595523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332c82-acfd-41d2-9f18-fce7cc2476cc None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:14.595797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3332c82-acfd-41d2-9f18-fce7cc2476cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:14.678382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a3332c82-acfd-41d2-9f18-fce7cc2476cc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:14.679659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a3332c82-acfd-41d2-9f18-fce7cc2476cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:14.822541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 699/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:15.188538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e920-d130-44e5-9843-3981889ae158 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:15.188979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e920-d130-44e5-9843-3981889ae158 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:15.189258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551e920-d130-44e5-9843-3981889ae158 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:15.190899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 692/1391] 10.222.0.26 () {54 vars in 966 bytes} [Mon Sep 12 23:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:15.197698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839abe68-aae1-4093-a0d6-818539223473 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:15.198512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839abe68-aae1-4093-a0d6-818539223473 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:15.198658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839abe68-aae1-4093-a0d6-818539223473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:15.252411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:15.288932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-839abe68-aae1-4093-a0d6-818539223473 None None] MFA Rules not processed for user `afd5af60bde74ab0a3e8f3f2d17ed8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:15.290630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-839abe68-aae1-4093-a0d6-818539223473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:15.405170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 700/1392] 10.222.0.26 () {58 vars in 1063 bytes} [Mon Sep 12 23:46:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:15.415055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e920-d130-44e5-9843-3981889ae158 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:15.416645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0551e920-d130-44e5-9843-3981889ae158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:15.543574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551e920-d130-44e5-9843-3981889ae158 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:15.584685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:15.626802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:15.627666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:15.627774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:15.627969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:15.629640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:15.750282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:15.751691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b03a-57c1-488d-972e-2408f882d486 service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:15.771982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 693/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:46:16.392469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead16e7-a840-41c4-959e-23436c84b4b6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:16.393309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead16e7-a840-41c4-959e-23436c84b4b6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:16.393479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead16e7-a840-41c4-959e-23436c84b4b6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:16.396176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 701/1394] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:46:16.408708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886848ad-8180-45ab-be2d-814646449b31 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:16.409253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886848ad-8180-45ab-be2d-814646449b31 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:16.409761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886848ad-8180-45ab-be2d-814646449b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:16.491602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-886848ad-8180-45ab-be2d-814646449b31 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:16.493113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-886848ad-8180-45ab-be2d-814646449b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:16.513261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:16.612727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f09f-9325-4c4a-99d6-0028f4faa16b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:16.613026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f09f-9325-4c4a-99d6-0028f4faa16b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:16.613431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28f09f-9325-4c4a-99d6-0028f4faa16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:16.629881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 694/1395] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:16.700513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fd28f09f-9325-4c4a-99d6-0028f4faa16b None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:16.702001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fd28f09f-9325-4c4a-99d6-0028f4faa16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:16.816181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:16.829070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 702/1396] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:46:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:16.906118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:16.906430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:16.906661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:16.908116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 695/1397] 10.222.0.26 () {54 vars in 943 bytes} [Mon Sep 12 23:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:16.918997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:16.919876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:16.920353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:16.948703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:16.950403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:17.045523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] MFA Rules not processed for user `3cb125ada8d64bc49ff99383d05aa69e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:17.047316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:17.149867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97283514-1c57-4509-9b8c-58c0dd0e7da3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:17.153300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:17.200204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 703/1398] 10.222.0.26 () {58 vars in 1040 bytes} [Mon Sep 12 23:46:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:17.218611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:17.228815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c4a1-1361-42b3-a253-b8f5b2b0e658 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:17.307721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:17.330296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:17.330920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:17.331195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:17.331501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:17.333421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:17.381585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:17.382212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:17.382485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:17.382753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:17.384687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:17.463995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:17.465951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee43f8-a928-4faa-a73f-003ab4faca1a service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:17.485927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 696/1399] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:46:17.489371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:17.490994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35041f6-0112-4840-801a-33bc087f056f service swift] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:17.502714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 704/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:46:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:46:19.916616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:19.917208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:19.917208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:19.918524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 697/1401] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:46:19.924453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76f05-f8f4-43e3-bb04-3a31046c0deb None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:19.925800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76f05-f8f4-43e3-bb04-3a31046c0deb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:19.926290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a76f05-f8f4-43e3-bb04-3a31046c0deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:20.028153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-09a76f05-f8f4-43e3-bb04-3a31046c0deb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:20.029601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-09a76f05-f8f4-43e3-bb04-3a31046c0deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:20.081139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:20.185351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 705/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:20.520971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:20.525085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:20.546946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:20.784886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f1756-11d3-4e58-8080-016436a9cf0e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:20.924393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:20.995684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:20.996669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:20.997113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:20.997550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:21.000544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:21.174750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:21.178000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0c416-c0a1-4744-af8f-989d52c3c750 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:21.200500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 698/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:23.576277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:23.576928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:23.576928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:23.578180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 706/1404] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:23.583712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486493-ca91-498f-8277-f40f5e5042f3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:23.584450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486493-ca91-498f-8277-f40f5e5042f3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:23.584450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486493-ca91-498f-8277-f40f5e5042f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:23.695847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0f486493-ca91-498f-8277-f40f5e5042f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:23.697314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f486493-ca91-498f-8277-f40f5e5042f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:23.758898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:23.835529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 699/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:46:24.129330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:24.132220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:24.390863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c843f8-83dd-4a92-b9a3-66e64c89ab9f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:24.430151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:24.654436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:24.655521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:24.655963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:24.656293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:24.659036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:24.841521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:24.844726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910a332-fdac-4a5b-8ca2-084d31d2ae5e service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:24.882460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 707/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:27.175690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:27.176457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:27.176457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:27.177647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 700/1407] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:27.183586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbece8e-87cc-4518-98c7-c0790288d30b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:27.184225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbece8e-87cc-4518-98c7-c0790288d30b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:27.184225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbece8e-87cc-4518-98c7-c0790288d30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:27.207330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:27.260703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7cbece8e-87cc-4518-98c7-c0790288d30b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:27.262013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7cbece8e-87cc-4518-98c7-c0790288d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:27.431356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 708/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:27.715896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:27.717508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:27.901596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847016e-18aa-443d-9f15-0a9ca5ce40d5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:27.936858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:28.071278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:28.071965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:28.072330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:28.072610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:28.074250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:28.192885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:28.195453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee49e7-9f77-48e6-9498-be9f984941b4 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:28.219001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 701/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:37.420170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=57 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:38.216431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=86 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:40.388163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d566e2-18dc-4e41-a985-b85a7081d5f1 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:40.388729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d566e2-18dc-4e41-a985-b85a7081d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:40.388729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d566e2-18dc-4e41-a985-b85a7081d5f1 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:40.389987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 709/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon Sep 12 23:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:40.395593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96c870-ba5d-48c2-8d31-c082f3d01edd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:40.396018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96c870-ba5d-48c2-8d31-c082f3d01edd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:40.396141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96c870-ba5d-48c2-8d31-c082f3d01edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:40.477916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0f96c870-ba5d-48c2-8d31-c082f3d01edd None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:40.479165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f96c870-ba5d-48c2-8d31-c082f3d01edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:40.601240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 702/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Sep 12 23:46:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:40.616442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e277df-8e26-48c6-bfbd-2298193ce66d None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:40.616720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e277df-8e26-48c6-bfbd-2298193ce66d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:40.616986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e277df-8e26-48c6-bfbd-2298193ce66d None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:40.618415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 710/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon Sep 12 23:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:40.623605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd510cc-1b3f-4993-93a4-bf0450d2f11b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:40.623889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd510cc-1b3f-4993-93a4-bf0450d2f11b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:40.624160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd510cc-1b3f-4993-93a4-bf0450d2f11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:40.703241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:40.705998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2fd510cc-1b3f-4993-93a4-bf0450d2f11b None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:40.707785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2fd510cc-1b3f-4993-93a4-bf0450d2f11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:40.830957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 703/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Sep 12 23:46:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:43.477318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:43.477861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:43.477861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:43.479257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:43.485139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b303-3574-4a58-ab69-44ff6308e711 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:43.485637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b303-3574-4a58-ab69-44ff6308e711 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:43.485736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84b303-3574-4a58-ab69-44ff6308e711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:43.560711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ff84b303-3574-4a58-ab69-44ff6308e711 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:43.561969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ff84b303-3574-4a58-ab69-44ff6308e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:43.630494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:43.676300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:43.965573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:43.967251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:44.129281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22af77-47fd-45ed-a10a-296a25d0a408 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:44.205227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:44.245488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:44.246102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:44.246382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:44.246647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:44.248315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:44.343029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:44.344684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ffe72-2e08-436f-bf0b-a041dec388a6 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:44.363472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 712/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:45.515302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:45.515903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:45.515903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:45.517215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:45.524458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231a1ee-ff86-4b20-b42b-019f25913a68 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:45.524966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231a1ee-ff86-4b20-b42b-019f25913a68 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:45.525085 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231a1ee-ff86-4b20-b42b-019f25913a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:45.612613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7231a1ee-ff86-4b20-b42b-019f25913a68 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:45.613853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7231a1ee-ff86-4b20-b42b-019f25913a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:45.643774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:45.721574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:45.876957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:45.878811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:46.041246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ae36b-db9e-489a-8287-6611250df08d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:46.088590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:46.163769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:46.164741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:46.165015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:46.165353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:46.167002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:46.261671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:46.263373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c281a3b-318a-4819-a7fe-62492ac44c65 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:46.283195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 706/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:47.449286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:47.449872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:47.449872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:47.451120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:47.458136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc81e-6534-4a2c-ad9a-39843e1f8ad5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:47.458701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc81e-6534-4a2c-ad9a-39843e1f8ad5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:47.458701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc81e-6534-4a2c-ad9a-39843e1f8ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:47.514980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:47.540327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-22fbc81e-6534-4a2c-ad9a-39843e1f8ad5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:47.541575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-22fbc81e-6534-4a2c-ad9a-39843e1f8ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:47.652504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:47.806437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:47.808143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:47.961256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498075fb-d7ce-4598-80e5-3f9f96bc4d3d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:47.974035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:48.080848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:48.081475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:48.081779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:48.082048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:48.083720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:48.179197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:48.180968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f2729-4ac0-41c8-87c9-7f0334122775 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:48.199871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 715/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:49.319499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:49.320052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:49.320052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:49.321995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:49.329140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5c79-587c-4293-8f25-7395169ab603 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:49.329650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5c79-587c-4293-8f25-7395169ab603 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:49.329650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5c79-587c-4293-8f25-7395169ab603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:49.353201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:49.420764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0e5c5c79-587c-4293-8f25-7395169ab603 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:49.422164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0e5c5c79-587c-4293-8f25-7395169ab603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:49.556046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:49.705626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:49.707473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:49.827388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:49.989343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cf434-c361-419a-8f52-f0ecf59efd6a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:50.109817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:50.110438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:50.110716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:50.110973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:50.112614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:50.193433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:50.208924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:50.210522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0c42b-b968-4fdf-acd8-84766875a4a5 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:50.229290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 709/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:51.373335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:51.373894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:51.373894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:51.375240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:51.384685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2096-06b1-408d-aea6-b58d936ae209 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:51.384685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2096-06b1-408d-aea6-b58d936ae209 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:51.384685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d2096-06b1-408d-aea6-b58d936ae209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:51.472040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-dc0d2096-06b1-408d-aea6-b58d936ae209 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:51.473285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dc0d2096-06b1-408d-aea6-b58d936ae209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:51.574095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:51.727175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:51.728918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:51.818903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:51.889311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959fa58-5d1f-4e03-bb9a-de57f6ec1b2b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:52.009604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:52.010129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:52.010407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:52.010678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:52.012370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:52.067671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:52.110792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:52.112454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec03f87-3462-4c25-843a-b4a19af3a96b service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:52.135780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 718/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:53.217147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:53.217788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:53.217788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:53.219084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:53.226882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c58cb-53cf-4035-96fd-e3b2b48dcbaa None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:53.227445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c58cb-53cf-4035-96fd-e3b2b48dcbaa None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:53.227445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c58cb-53cf-4035-96fd-e3b2b48dcbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:53.307894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5d7c58cb-53cf-4035-96fd-e3b2b48dcbaa None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:53.309127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5d7c58cb-53cf-4035-96fd-e3b2b48dcbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:53.409877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:53.558683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:53.560331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:53.613109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:53.715520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac2dcc-d223-4837-a3b7-b9f5ee9d1f81 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:53.845430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:53.846074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:53.846330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:53.846597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:53.848288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:53.856071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:53.945041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:53.946754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2377094-69cb-4154-9844-8b72b3030c21 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:53.965109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 712/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:55.026914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:55.027473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:55.027473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:55.029916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:55.035730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67244dfd-d21a-402a-bb00-3f6caef2cdfc None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:55.036254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67244dfd-d21a-402a-bb00-3f6caef2cdfc None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:55.036254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67244dfd-d21a-402a-bb00-3f6caef2cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:55.118776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-67244dfd-d21a-402a-bb00-3f6caef2cdfc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:55.120048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-67244dfd-d21a-402a-bb00-3f6caef2cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:55.194320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:55.224636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:55.376140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:55.377693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:55.396489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:55.546697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dbe73-ab71-47a0-80b6-2e036629cbaa None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:55.639704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:55.668162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:55.668764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:55.669014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:55.669294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:55.670942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:55.769022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:55.770630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a0ae9-b83b-4402-8e8e-951c64af4c89 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:55.789976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 721/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:56.835101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:56.835712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:56.835712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:56.837177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:56.844561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474b-8c5b-4c9c-8e5c-c5a7be4a66c5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:56.845173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474b-8c5b-4c9c-8e5c-c5a7be4a66c5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:56.845173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474b-8c5b-4c9c-8e5c-c5a7be4a66c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:56.941691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-8817474b-8c5b-4c9c-8e5c-c5a7be4a66c5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:56.943149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8817474b-8c5b-4c9c-8e5c-c5a7be4a66c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:56.995069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:57.053695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:57.203593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:57.205057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:57.361705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48066af-7801-4d81-8308-2e5b029915c6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:57.418023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:57.479758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:57.480245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:57.480521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:57.480785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:57.482399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:57.589260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:57.590863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fb0f-8715-4cbe-9eab-f19e729aa855 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:57.612318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 715/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:46:58.678419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:58.679028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:58.679028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:58.680361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:46:58.687508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483df00-1b90-458b-b773-a61260d18b1f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:58.688021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483df00-1b90-458b-b773-a61260d18b1f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:58.688118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483df00-1b90-458b-b773-a61260d18b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:58.780338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0483df00-1b90-458b-b773-a61260d18b1f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:46:58.781584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0483df00-1b90-458b-b773-a61260d18b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:58.783722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:58.884156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:46:59.032106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:46:59.033946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:59.199305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6459c0-ee65-4dae-8999-b7d90e413d9d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:46:59.228716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:46:59.321447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:46:59.322067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:46:59.322346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:46:59.322639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:46:59.324283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:46:59.422047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:46:59.423736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e50e8e-4e37-4781-9e94-eb57523324b7 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:46:59.442560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 724/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:01.544989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:01.545592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:01.545592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:01.546778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:01.552439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c329eb-3f1f-4ee8-946d-4c4534b3d06a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:01.552970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c329eb-3f1f-4ee8-946d-4c4534b3d06a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:01.552970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c329eb-3f1f-4ee8-946d-4c4534b3d06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:01.598979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:01.632070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f9c329eb-3f1f-4ee8-946d-4c4534b3d06a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:01.633311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f9c329eb-3f1f-4ee8-946d-4c4534b3d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:01.741709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:02.024411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:02.025987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:02.146831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb333d-26b3-4f7d-b99d-4df7e473ea6b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:02.185512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:02.226896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:02.227619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:02.227960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:02.228223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:02.229824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:02.339072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:02.340573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a189b-ad48-48f6-b8cf-56af6dee91a4 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:02.359560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 718/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:04.726645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:04.727302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:04.727302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:04.728613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:04.734032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f41c-56d2-4dfc-9656-61698073ea79 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:04.734469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f41c-56d2-4dfc-9656-61698073ea79 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:04.734562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55f41c-56d2-4dfc-9656-61698073ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:04.807083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3e55f41c-56d2-4dfc-9656-61698073ea79 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:04.808340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3e55f41c-56d2-4dfc-9656-61698073ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:04.822027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:04.911885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:05.192930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:05.194492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:05.316216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad01f5-8608-44c8-b345-e57aa447a738 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:05.330575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:05.397040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:05.397639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:05.397918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:05.398193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:05.399871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:05.530372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:05.532294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd6c6e-3cbc-4eb6-8029-4aed9ccd2ad8 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:05.551611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 727/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:08.055451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:08.056205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:08.056205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:08.057513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:08.063030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f4d5a-2f0e-48bd-ba94-afeee4806f92 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:08.063503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f4d5a-2f0e-48bd-ba94-afeee4806f92 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:08.063626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f4d5a-2f0e-48bd-ba94-afeee4806f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:08.117350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:08.137771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-eb0f4d5a-2f0e-48bd-ba94-afeee4806f92 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:08.139020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-eb0f4d5a-2f0e-48bd-ba94-afeee4806f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:08.260928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:08.542046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:08.543664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:08.655819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc6be1-3241-4ef8-aef0-2b44a11e9bb1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:08.698509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:08.736512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:08.737126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:08.737394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:08.737658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:08.739276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:08.856434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:08.857918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a21eb-1913-4c48-be43-85498bf4da4e service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:08.876814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 721/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:11.190437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:11.191081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:11.191081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:11.192336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:11.199245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365365-0a75-4cc8-8de6-8e35d54825fd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:11.199721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365365-0a75-4cc8-8de6-8e35d54825fd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:11.199961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365365-0a75-4cc8-8de6-8e35d54825fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:11.275984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-db365365-0a75-4cc8-8de6-8e35d54825fd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:11.277284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-db365365-0a75-4cc8-8de6-8e35d54825fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:11.304059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:11.390835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:11.676339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:11.678296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:11.785358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24903c3-5367-4b27-8705-6d911767d7db None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:11.807575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:11.861483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:11.862088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:11.862365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:11.862633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:11.864286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:11.980809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:11.982260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81433868-6ec2-42a8-88e2-b47fafccaa3b service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:12.000716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 730/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:14.240493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:14.241090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:14.241090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:14.242681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:14.248693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d553f-7976-4a12-80fc-85f46211381d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:14.249156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d553f-7976-4a12-80fc-85f46211381d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:14.249410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d553f-7976-4a12-80fc-85f46211381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:14.318060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:14.321444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2c6d553f-7976-4a12-80fc-85f46211381d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:14.323250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2c6d553f-7976-4a12-80fc-85f46211381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:14.449205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:14.736730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:14.738288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:14.848649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f343cc-d658-49b8-b125-a53351d01697 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:14.899120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:14.925613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:14.926226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:14.926503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:14.926772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:14.928431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:15.044601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:15.045976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b585c-2ed7-4756-942a-bb54aa03003d service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:15.066283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 724/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:17.276177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:17.276805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:17.276805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:17.278074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:17.283711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6751-0a92-451c-8995-e8d113dda87b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:17.284160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6751-0a92-451c-8995-e8d113dda87b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:17.284272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6751-0a92-451c-8995-e8d113dda87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:17.354221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c64d6751-0a92-451c-8995-e8d113dda87b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:17.355380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c64d6751-0a92-451c-8995-e8d113dda87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:17.385818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:17.459968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:47:17.740281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:17.741859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:17.855009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c0664-34e2-41ca-a09b-b14622b18a04 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:17.885809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:17.935399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:17.936019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:17.936297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:17.936570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:17.938187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:18.056609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:18.058018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab56763-a0c6-4771-884b-bd21d05bfad4 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:18.078887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 733/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:20.338429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:20.338967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:20.338967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:20.340411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:20.346407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d10c-516f-46ec-8b91-ef0bbd7febb3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:20.346844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d10c-516f-46ec-8b91-ef0bbd7febb3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:20.346944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d10c-516f-46ec-8b91-ef0bbd7febb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:20.422848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5b32d10c-516f-46ec-8b91-ef0bbd7febb3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:20.424142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5b32d10c-516f-46ec-8b91-ef0bbd7febb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:20.433185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:20.536239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:20.817251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:20.818838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:20.928801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb6ff-8da1-47f1-973c-82471e21a0e9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:20.989020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:21.007869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:21.008601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:21.008841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:21.009106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:21.010737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:21.122428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:21.123941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac4ce-0fc8-4f42-8a54-e5623893e80b service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:21.143082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 727/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:23.366550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:23.367098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:23.367098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:23.368560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:23.374406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46efa0b-e42a-4c87-9dda-0419c7e8e15b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:23.374901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46efa0b-e42a-4c87-9dda-0419c7e8e15b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:23.374901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46efa0b-e42a-4c87-9dda-0419c7e8e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:23.443892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b46efa0b-e42a-4c87-9dda-0419c7e8e15b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:23.445117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b46efa0b-e42a-4c87-9dda-0419c7e8e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:23.485745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:23.547963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:23.836322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:23.837906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:23.951182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1a10-f661-4b85-a74a-0d5c87d01d14 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:23.990789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:24.028868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:24.029479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:24.029751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:24.030015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:24.031644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:24.149786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:24.151256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667090a-b1e5-4c77-8d60-eacdf42ac6c8 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:24.170740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 736/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:26.348138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:26.348732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:26.348732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:26.349914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:47:26.355659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841965dc-4962-4bcb-a507-f2b871d8f1e2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:26.356158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841965dc-4962-4bcb-a507-f2b871d8f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:26.356158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841965dc-4962-4bcb-a507-f2b871d8f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:26.430999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-841965dc-4962-4bcb-a507-f2b871d8f1e2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:26.432370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-841965dc-4962-4bcb-a507-f2b871d8f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:26.451187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:26.534278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:47:26.817821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:26.819435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:26.933647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ac7ed-bf0f-4481-8638-81684bdcf62c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:27.018294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:27.026300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:27.026918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:27.027196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:27.027492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:27.029113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:27.145326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:27.146782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcaf24-dde9-471a-a0d2-c19b46658e58 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:27.166535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 730/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:29.337042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:29.337603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:29.337603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:29.338905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:29.344836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d744-0a41-4b88-b1d8-219150140515 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:29.345325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d744-0a41-4b88-b1d8-219150140515 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:29.345439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6d744-0a41-4b88-b1d8-219150140515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:29.423796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3de6d744-0a41-4b88-b1d8-219150140515 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:29.425048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3de6d744-0a41-4b88-b1d8-219150140515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:29.475583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:29.526032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:29.817311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:29.818877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:29.930767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055d301-0981-444d-bbfe-df52e4b5ece2 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:29.981533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:30.018910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:30.019574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:30.019832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:30.020099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:30.021703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:30.145679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:30.147122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f924e-1d18-4cce-8910-c5aec154719d service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:30.167029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 739/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:39.524450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=22 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:40.164395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=66 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:50.132846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfcbf4-2304-46a8-8362-4ab73b0fd555 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:50.133516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfcbf4-2304-46a8-8362-4ab73b0fd555 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:50.133626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfcbf4-2304-46a8-8362-4ab73b0fd555 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:50.135789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 732/1471] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:50.143250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaeb31-0c22-45f0-8f0e-05996b5f2dc9 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:50.143807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaeb31-0c22-45f0-8f0e-05996b5f2dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:50.143807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaeb31-0c22-45f0-8f0e-05996b5f2dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:50.222592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-efaaeb31-0c22-45f0-8f0e-05996b5f2dc9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:50.223869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-efaaeb31-0c22-45f0-8f0e-05996b5f2dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:50.349646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 740/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:47:50.653625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:50.653864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:50.654150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:50.655602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 733/1473] 10.222.0.26 () {54 vars in 966 bytes} [Mon Sep 12 23:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:50.660973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095af90-0bb0-409d-b77a-7a50a84e9492 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:50.661260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095af90-0bb0-409d-b77a-7a50a84e9492 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:50.661566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095af90-0bb0-409d-b77a-7a50a84e9492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:50.741893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f095af90-0bb0-409d-b77a-7a50a84e9492 None None] MFA Rules not processed for user `afd5af60bde74ab0a3e8f3f2d17ed8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:50.743247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f095af90-0bb0-409d-b77a-7a50a84e9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:50.767347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:50.827031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 741/1474] 10.222.0.26 () {58 vars in 1063 bytes} [Mon Sep 12 23:47:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:50.834936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:50.836768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:50.946899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e7ac7-8f0c-4aaa-9f35-8bc61ef937ae None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:51.029191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:51.029794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:51.030068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:51.030337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:51.032005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:51.090898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:51.149620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:51.151104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae785526-e63e-4f71-abe7-cc8940cb35f2 service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:51.172832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 734/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:53.383475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:53.384016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:53.384016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:53.385851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 742/1476] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:47:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:53.391992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44e6d-6346-431c-9574-fb95931be765 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:53.392594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44e6d-6346-431c-9574-fb95931be765 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:53.392594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44e6d-6346-431c-9574-fb95931be765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:53.468046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6db44e6d-6346-431c-9574-fb95931be765 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:53.469321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6db44e6d-6346-431c-9574-fb95931be765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:53.583936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 735/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:53.853172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:53.854750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:53.992374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a3afa-5774-4eb4-af53-bba72172a1e5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:54.047057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:54.094936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:54.095591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:54.095868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:54.096140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:54.096780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:54.098598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:47:54.098598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:47:54.098598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:47:54.099768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4796c-1c51-4183-90ba-4510792b96d1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:54.119658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 743/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Sep 12 23:47:53 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:47:56.298762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:56.299373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:56.299373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:56.300705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 736/1479] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:56.306569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508e5f3-ff79-4509-9f7e-4027c805775b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:56.307069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508e5f3-ff79-4509-9f7e-4027c805775b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:56.307069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508e5f3-ff79-4509-9f7e-4027c805775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:56.381127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1508e5f3-ff79-4509-9f7e-4027c805775b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:56.382605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1508e5f3-ff79-4509-9f7e-4027c805775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:56.496951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 744/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:47:56.795643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:47:56.797309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:56.807795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:56.954795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2864b-5029-402b-86d6-51cbd919694d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:47:57.039473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:57.076816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:47:57.077438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:57.077715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:57.077970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:57.079688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:57.174105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:47:57.175766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc546c-fdfa-4851-b74f-c58d9fa9994d service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:47:57.194303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 737/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:47:59.557418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:59.557978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:59.557978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:59.559265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 745/1482] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:47:59.564751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3323-80d9-4e08-89f9-87c25790e1f5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:47:59.565208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3323-80d9-4e08-89f9-87c25790e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:47:59.565313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3323-80d9-4e08-89f9-87c25790e1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:47:59.639022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-eb3e3323-80d9-4e08-89f9-87c25790e1f5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:47:59.640283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-eb3e3323-80d9-4e08-89f9-87c25790e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:47:59.698543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:47:59.760210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 738/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:48:00.054011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:48:00.055715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:00.082715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:00.170477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02a64d-6428-4f82-bc19-09bfde398410 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:48:00.250088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:48:00.250694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:00.250972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:00.251260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:00.252894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:00.461432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:00.500046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:48:00.501519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acf85e-c007-4d09-a79a-764fb6d4372e service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:48:00.521019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 746/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:48:02.718790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:02.719410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:02.719410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:02.720740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 739/1485] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:48:02.726870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd1fe-b38d-4846-b349-63ffb8a15239 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:02.727427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd1fe-b38d-4846-b349-63ffb8a15239 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:02.727602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd1fe-b38d-4846-b349-63ffb8a15239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:02.801873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f54cd1fe-b38d-4846-b349-63ffb8a15239 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:48:02.803107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f54cd1fe-b38d-4846-b349-63ffb8a15239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:02.910519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 747/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:48:03.202491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:48:03.204076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:03.313045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee76ef-07f6-472e-a667-b047097886e0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:48:03.388458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:03.390368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:48:03.391286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:03.391584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:03.392162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:03.394473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:03.523940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:48:03.525404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dc25a-dc64-4d53-b1e3-481ed5f4767b service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:48:03.544303 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 740/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:48:05.786691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7383bef-577f-432d-9d81-72719540e255 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:05.787333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7383bef-577f-432d-9d81-72719540e255 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:05.787333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7383bef-577f-432d-9d81-72719540e255 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:05.788968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 748/1488] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:48:05.795583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52d24-404d-47f3-b32a-4ad9d74a3b1d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:05.796131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52d24-404d-47f3-b32a-4ad9d74a3b1d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:05.796131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52d24-404d-47f3-b32a-4ad9d74a3b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:05.870671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9ae52d24-404d-47f3-b32a-4ad9d74a3b1d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:48:05.871933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9ae52d24-404d-47f3-b32a-4ad9d74a3b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:05.930961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:05.982098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 741/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:48:06.276763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7383bef-577f-432d-9d81-72719540e255 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:48:06.278422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e7383bef-577f-432d-9d81-72719540e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:06.319578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:06.407312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7383bef-577f-432d-9d81-72719540e255 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:48:06.506741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:48:06.507410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:06.507692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:06.507970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:06.509780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:06.605726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:06.639170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:48:06.640690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d9082-c465-4dce-ac3b-b46184afc977 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:48:06.661540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 749/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:48:09.113315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:09.113890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:09.113890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:09.115271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 742/1491] 10.222.0.26 () {54 vars in 927 bytes} [Mon Sep 12 23:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:48:09.122322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458eb5c-baf9-41f6-8e2d-beccb53220cd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:09.122804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458eb5c-baf9-41f6-8e2d-beccb53220cd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:09.122898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458eb5c-baf9-41f6-8e2d-beccb53220cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:09.192966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9458eb5c-baf9-41f6-8e2d-beccb53220cd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:48:09.194173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9458eb5c-baf9-41f6-8e2d-beccb53220cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:09.301144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 750/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Sep 12 23:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:48:09.598300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:48:09.601154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:09.723664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180b0bd-24fe-456a-a49b-119a9ea46e71 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:48:09.809499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:48:09.810089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:48:09.810937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:48:09.810937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:48:09.812335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:48:09.824738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:09.943532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:48:09.944906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393fb3-fa91-41ec-bc6b-a9962609a794 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:48:09.966170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 743/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:48:19.299145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=82 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:48:19.963363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=50 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:22.889099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:22.889812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:22.889812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:22.981011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:49:22.982278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:23.089250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 751/1494] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:49:23.098325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:23.098920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:23.098920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:23.182827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:49:23.184004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:23.299400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 744/1495] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:49:23.304578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:23.305855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:23.318320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cb24b-bcbb-424e-ad2f-9cca3cdf1e80 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:23.399233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:23.440301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:23.440839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:23.441116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:23.441398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:23.442028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:49:23.442326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:23.443381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:23.443381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:49:23.443381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:49:23.444501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:23.457360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 752/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:49:23.466305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:23.476276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762ebfd-53bd-43f4-b6b9-bdec528f0c93 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:23.543409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:23.579649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:23.580262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:23.580528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:23.580782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:23.581410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:23.583176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:23.583176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:49:23.583176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:49:23.584388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:23.645164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 745/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:49:23 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:49:23.650203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:23.651914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:23.689048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472ff55-3712-4a5b-9b6f-6d068f4193ac admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:23.803745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:23.804271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:23.804543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:23.804821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:23.805482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:23.806536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:23.806536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:49:23.806536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:49:23.807804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:23.860735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 753/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:49:23 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:49:23.865241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:23.866821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:23.936639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2277d5-9bc8-4e27-a194-99c600d03cc8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:24.026649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:24.044975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:24.045600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.045870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.046138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.046786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:24.047873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:24.047873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:49:24.047873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:49:24.048789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:24.063128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 746/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:49:24.068643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:24.071131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:24.083138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1534fc-29ae-46c7-a137-c4b6f4427d75 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:24.109815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:24.184010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:24.184625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.184898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.185162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.185839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:24.186859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:24.186859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:49:24.186859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:49:24.187773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54e71b-a80c-4f65-9a03-73caa8fdadf1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:24.200924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 754/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:49:24.205346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:24.206867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:24.242858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49508ed-0d5e-4cea-b9ab-4b9771660127 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:24.344135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:24.344806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.345081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.345349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] PATH_INFO: `/v3/projects/cdf7c3e8e3f047c8aa61c72942630a94/users/bd664ae971bc4d2eb19b797bde1c7be5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.369854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf7c3e8e3f047c8aa61c72942630a94, user_id=bd664ae971bc4d2eb19b797bde1c7be5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:24.370888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:24.370888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:49:24.370888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:49:24.372859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15732c90-5983-460e-b67b-ba08faf6729d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:24.399510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 747/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:49:24 2022] PUT /identity/v3/projects/cdf7c3e8e3f047c8aa61c72942630a94/users/bd664ae971bc4d2eb19b797bde1c7be5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:49:24.412497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568180-de4a-487e-8ed5-5b6642559958 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.412818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568180-de4a-487e-8ed5-5b6642559958 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.413088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e568180-de4a-487e-8ed5-5b6642559958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.471142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0e568180-de4a-487e-8ed5-5b6642559958 None None] MFA Rules not processed for user `bd664ae971bc4d2eb19b797bde1c7be5`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:49:24.472395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0e568180-de4a-487e-8ed5-5b6642559958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:24.552544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 755/1502] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:49:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:49:24.566616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.566900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.567170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.568578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 748/1503] 10.222.0.26 () {54 vars in 964 bytes} [Mon Sep 12 23:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:49:24.574562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee67a5-d8db-44e1-9320-d38b6cf20b8a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.574927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee67a5-d8db-44e1-9320-d38b6cf20b8a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.575287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee67a5-d8db-44e1-9320-d38b6cf20b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.611292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:24.657987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6eee67a5-d8db-44e1-9320-d38b6cf20b8a None None] MFA Rules not processed for user `65d52c53e1244d25b0e16374101ea139`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:49:24.659256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6eee67a5-d8db-44e1-9320-d38b6cf20b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:24.749289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 756/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Sep 12 23:49:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:49:24.757412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:24.759076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:24.813371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:24.869564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bd42b-6660-4da5-aefc-11825231d113 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:24.950871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:24.951503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:24.951776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:24.952047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:24.953661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:25.064665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:25.066168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:25.069962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:25.085586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 749/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:49:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:49:25.131774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d98e-0ac5-4347-ae5f-19560ea37f1f None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:25.132226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d98e-0ac5-4347-ae5f-19560ea37f1f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:25.132308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d98e-0ac5-4347-ae5f-19560ea37f1f None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:25.133606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 757/1506] 10.222.0.26 () {52 vars in 830 bytes} [Mon Sep 12 23:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:49:25.137448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:25.138696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:25.246606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f9de7-bda1-42f3-ac35-8775628bcdb2 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:25.343005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:25.343697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:25.343976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:25.344245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] PATH_INFO: `/v3/users/bd664ae971bc4d2eb19b797bde1c7be5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:25.358224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd664ae971bc4d2eb19b797bde1c7be5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:25.359421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:25.359421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:49:25.359421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:49:25.361840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:25.371767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:25.441107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd664ae971bc4d2eb19b797bde1c7be5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:25.441642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd664ae971bc4d2eb19b797bde1c7be5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:25.442059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd664ae971bc4d2eb19b797bde1c7be5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:25.456753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd664ae971bc4d2eb19b797bde1c7be5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:25.483055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd664ae971bc4d2eb19b797bde1c7be5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:25.498980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 750/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:49:25 2022] DELETE /identity/v3/users/bd664ae971bc4d2eb19b797bde1c7be5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:49:25.503742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:25.504046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:25.504314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:25.565449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:49:25.566682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:25.646120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:25.658885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 758/1508] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:49:25.669331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:25.670614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:25.784284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e81be-9058-445d-846d-fa897e8c7561 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:25.851463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:25.873758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:25.874390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:25.874666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:25.874937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:25.876581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:25.994217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:25.995596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d95e0-efdb-433e-8e2e-ab5de56b8bab service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:26.014102 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 751/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:49:26.358906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:49:26.360279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:49:26.372917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f632-fb9d-4821-a6b8-771a0cfb2831 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:49:26.482609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:49:26.483264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:49:26.483578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:49:26.483844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] PATH_INFO: `/v3/projects/cdf7c3e8e3f047c8aa61c72942630a94` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:49:26.495999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf7c3e8e3f047c8aa61c72942630a94)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:49:26.497048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:49:26.497048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:49:26.497048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:49:26.498261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:49:26.564948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf7c3e8e3f047c8aa61c72942630a94', 'request_id': 'req-9fb7da22-4622-4ebe-bdab-5222bae9269b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:26.578152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf7c3e8e3f047c8aa61c72942630a94', 'request_id': 'req-9fb7da22-4622-4ebe-bdab-5222bae9269b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:26.585095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf7c3e8e3f047c8aa61c72942630a94', 'request_id': 'req-9fb7da22-4622-4ebe-bdab-5222bae9269b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:26.613856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] The token cache is being invalidate because project cdf7c3e8e3f047c8aa61c72942630a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:49:26.614144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:26.614680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9fb7da22-4622-4ebe-bdab-5222bae9269b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:49:26.616492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 759/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:49:26 2022] DELETE /identity/v3/projects/cdf7c3e8e3f047c8aa61c72942630a94 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:49:36.011443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=62 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:49:36.611467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:50:58.167817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:50:58.168408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:50:58.168408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:50:58.169619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:50:58.174784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67f1c-eba1-4836-b5d2-72b4ad2b0975 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:50:58.175307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67f1c-eba1-4836-b5d2-72b4ad2b0975 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:50:58.175307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67f1c-eba1-4836-b5d2-72b4ad2b0975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:50:58.261974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-64b67f1c-eba1-4836-b5d2-72b4ad2b0975 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:50:58.263248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-64b67f1c-eba1-4836-b5d2-72b4ad2b0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:50:58.377529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:50:58.531476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:50:58.533865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:50:58.672478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24559e85-6811-4e25-9468-f896140eb8b7 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:50:58.678629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:50:58.679266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:50:58.679549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:50:58.679817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:50:58.689041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:50:58.690673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a955b0-1ea2-44c1-86f4-e35c1fd755d8 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:50:58.701207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 753/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:50:59.821435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046a6a-80ff-4468-b579-679d4821e849 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:50:59.822027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046a6a-80ff-4468-b579-679d4821e849 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:50:59.822027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046a6a-80ff-4468-b579-679d4821e849 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:50:59.823428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:50:59.832371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064f24-2737-4b31-b50a-c3bf438c796b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:50:59.832936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064f24-2737-4b31-b50a-c3bf438c796b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:50:59.833045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064f24-2737-4b31-b50a-c3bf438c796b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:50:59.854586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-66064f24-2737-4b31-b50a-c3bf438c796b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:50:59.855851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-66064f24-2737-4b31-b50a-c3bf438c796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:50:59.865601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:00.008091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e53d-ada5-4f94-8056-951c9f0ec85c None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:00.008460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e53d-ada5-4f94-8056-951c9f0ec85c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:00.008684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e53d-ada5-4f94-8056-951c9f0ec85c None None] PATH_INFO: `/v3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:00.009975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 762/1516] 10.222.0.18 () {54 vars in 925 bytes} [Mon Sep 12 23:51:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:51:00.017960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86701d7f-5970-4df5-88c3-f2dfca5b36c2 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:00.018265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86701d7f-5970-4df5-88c3-f2dfca5b36c2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:00.018532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86701d7f-5970-4df5-88c3-f2dfca5b36c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:00.019825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9e53d-ada5-4f94-8056-951c9f0ec85c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:00.034608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9e53d-ada5-4f94-8056-951c9f0ec85c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:00.041059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:00.041677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:00.041977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:00.042248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:00.051568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:00.053337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64613ff7-d7ad-4764-b4f7-1fa29d04eaa8 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:00.064206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 763/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:00.086734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-86701d7f-5970-4df5-88c3-f2dfca5b36c2 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:00.088049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-86701d7f-5970-4df5-88c3-f2dfca5b36c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:00.152784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:51:00.176197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 755/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Sep 12 23:51:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:03.836372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d475-3047-43f2-b27f-36ffe47fa887 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:03.837051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d475-3047-43f2-b27f-36ffe47fa887 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:03.837051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d475-3047-43f2-b27f-36ffe47fa887 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:03.838336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 764/1519] 10.222.0.26 () {54 vars in 961 bytes} [Mon Sep 12 23:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:03.843763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2b6a-e17d-4ae5-abc6-7dff1497b780 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:03.844237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2b6a-e17d-4ae5-abc6-7dff1497b780 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:03.844339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2b6a-e17d-4ae5-abc6-7dff1497b780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:03.866897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3bfb2b6a-e17d-4ae5-abc6-7dff1497b780 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:03.868143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3bfb2b6a-e17d-4ae5-abc6-7dff1497b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:03.877701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 756/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Sep 12 23:51:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:03.885721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d475-3047-43f2-b27f-36ffe47fa887 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:03.899794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d475-3047-43f2-b27f-36ffe47fa887 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:03.905259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:03.905867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:03.906144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:03.906414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:03.916212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:03.917790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f3a94-cc71-4c5c-8eaa-e9d6f61960e5 service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:03.928940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 765/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:11.602716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f4df-f337-4cf3-8d4c-b2f7e54d34f0 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:11.603352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f4df-f337-4cf3-8d4c-b2f7e54d34f0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:11.603352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f4df-f337-4cf3-8d4c-b2f7e54d34f0 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:11.604665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:11.611405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a234a8-078e-41f0-b34d-6e45457aeef5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:11.611860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a234a8-078e-41f0-b34d-6e45457aeef5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:11.611953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a234a8-078e-41f0-b34d-6e45457aeef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:11.634304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e4a234a8-078e-41f0-b34d-6e45457aeef5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:11.635571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e4a234a8-078e-41f0-b34d-6e45457aeef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:11.646294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:11.804014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914f4df-f337-4cf3-8d4c-b2f7e54d34f0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:11.820000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914f4df-f337-4cf3-8d4c-b2f7e54d34f0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:11.825860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:11.826462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:11.826744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:11.827015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:11.835870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:11.837480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d17ab-846b-4402-a57b-fb6f244846b4 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:11.847208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 758/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:12.927348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb38-787b-4de5-a97e-19c10ed1b9aa None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:12.927988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb38-787b-4de5-a97e-19c10ed1b9aa None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:12.927988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ccb38-787b-4de5-a97e-19c10ed1b9aa None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:12.929285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 12 23:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:12.935857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee2f2-9eea-43ae-a86e-9663bdfcf641 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:12.936362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee2f2-9eea-43ae-a86e-9663bdfcf641 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:12.936362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee2f2-9eea-43ae-a86e-9663bdfcf641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:12.957227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-724ee2f2-9eea-43ae-a86e-9663bdfcf641 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:12.958461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-724ee2f2-9eea-43ae-a86e-9663bdfcf641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:12.967575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 12 23:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:13.119277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ccb38-787b-4de5-a97e-19c10ed1b9aa None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:13.135071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ccb38-787b-4de5-a97e-19c10ed1b9aa None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:13.141353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:13.141961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:13.142236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:13.142517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:13.152470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:13.154083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664220f-f6b7-4cb8-b5c6-fb203a08bfb2 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:13.165018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 768/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:17.290080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:17.290650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:17.290650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:17.292099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:17.297501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec909d-4ebe-4ae5-86c5-685030348591 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:17.297971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec909d-4ebe-4ae5-86c5-685030348591 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:17.298088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec909d-4ebe-4ae5-86c5-685030348591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:17.318155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6cec909d-4ebe-4ae5-86c5-685030348591 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:17.319451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6cec909d-4ebe-4ae5-86c5-685030348591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:17.327840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:17.617531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:17.619609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:17.714332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6ba73-519a-4426-b6c5-276793febe73 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:17.717961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:17.718575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:17.718978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:17.719277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:17.728408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:17.729877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684f57f-9e51-4d4b-a06c-810932a83f48 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:17.739941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 761/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:19.917859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:19.918461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:19.918461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:19.920074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:19.925397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc816-4632-45a2-9822-f4dea099f4ca None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:19.925882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc816-4632-45a2-9822-f4dea099f4ca None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:19.925989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc816-4632-45a2-9822-f4dea099f4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:19.946123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b70fc816-4632-45a2-9822-f4dea099f4ca None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:19.947422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b70fc816-4632-45a2-9822-f4dea099f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:19.955594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:51:20.236815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:20.238597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:20.254505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252cd61-bd9c-45cb-a0b2-c8e7921e1822 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:20.258615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:20.259323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:20.259675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:20.260044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:20.270352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:20.271829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ee05e-4fba-426c-a4ae-4229c566bc3f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:20.283405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 771/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:24.210990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:24.211632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:24.211632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:24.212919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:24.218565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9fa57-62fc-4f45-9c9e-87ecbb36a2f3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:24.219024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9fa57-62fc-4f45-9c9e-87ecbb36a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:24.219133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9fa57-62fc-4f45-9c9e-87ecbb36a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:24.227197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:51:24.241692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-02c9fa57-62fc-4f45-9c9e-87ecbb36a2f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:24.242916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-02c9fa57-62fc-4f45-9c9e-87ecbb36a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:24.251675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:24.553334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:24.555208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:24.638413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2103ef-eb53-4d36-8c94-237d54f2b824 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:24.642349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:24.642976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:24.643251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:24.643546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:24.653259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:24.654716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec6900-5f72-406e-bb0c-51a6e2c0e718 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:24.665006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 764/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:26.802577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:26.803186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:26.803186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:26.804510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 12 23:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:51:26.810873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319f814-ec4d-4bdd-9172-973040aa7c19 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:26.811423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319f814-ec4d-4bdd-9172-973040aa7c19 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:26.811423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319f814-ec4d-4bdd-9172-973040aa7c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:26.837372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0319f814-ec4d-4bdd-9172-973040aa7c19 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:51:26.838607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0319f814-ec4d-4bdd-9172-973040aa7c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:26.848046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 12 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:51:27.149296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:51:27.151470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:51:27.166311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9288b3b-8d65-42e0-93c6-67433b4bb4ba None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:51:27.170309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:51:27.170921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:51:27.171199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:51:27.171503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:51:27.181273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:51:27.182700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf941a-eb16-41f1-b93a-39a4c975761c service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:51:27.192431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 774/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:51:36.831943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:51:37.188453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=8 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:54.060731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e458-0ecf-438e-b373-7f4a2cfab015 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.062909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e458-0ecf-438e-b373-7f4a2cfab015 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.063485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633e458-0ecf-438e-b373-7f4a2cfab015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.100682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a633e458-0ecf-438e-b373-7f4a2cfab015 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.102338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a633e458-0ecf-438e-b373-7f4a2cfab015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.109323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.109922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.109922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.115491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 766/1540] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.125926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391935-30fb-4d90-a9ec-12b37e31d295 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.126556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391935-30fb-4d90-a9ec-12b37e31d295 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.127054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391935-30fb-4d90-a9ec-12b37e31d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.152981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.152981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.162999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-da391935-30fb-4d90-a9ec-12b37e31d295 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.165712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-da391935-30fb-4d90-a9ec-12b37e31d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.179917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 775/1541] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:52:54.184957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 767/1542] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:52:54.190893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.199282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.199940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.200479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.226830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ead9e-8fdb-486b-b566-05042a08bda7 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.235542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.236383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.236790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.237062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.237734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.238034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.239827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd2067-082e-4bc5-b7da-0e3b795e1967 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.240531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.242425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.263443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 776/1543] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.264959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.265239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.265506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.266177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 768/1544] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:52:54.273567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.297446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.298661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.300618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d2ff3-7c63-4500-b0fc-aacec6968ee4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.314990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.316082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.316872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.317387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.318432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.319706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.320063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 777/1545] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.320986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.321589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:52:54.321589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:52:54.321589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:52:54.322888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd6db-c216-47e2-871f-4b8b587c01ac admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.342994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed0f95-a406-40e0-a23f-98632c353b85 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.345106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 769/1546] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:52:54.346572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb34b-f072-41f8-b3f8-4ddaa2fcac97 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.347033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb34b-f072-41f8-b3f8-4ddaa2fcac97 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.347488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fb34b-f072-41f8-b3f8-4ddaa2fcac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.348434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.349065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.349334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.349613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.350325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.351394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:52:54.351394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:52:54.351394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:52:54.352594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a2995-ba4d-4dc7-ab0e-69e546d4fa86 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.370362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ea3fb34b-f072-41f8-b3f8-4ddaa2fcac97 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.372074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ea3fb34b-f072-41f8-b3f8-4ddaa2fcac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.380111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 770/1547] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.387894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13ce-fc2a-4caf-a2cb-ff6c63dd169e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.387894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13ce-fc2a-4caf-a2cb-ff6c63dd169e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.387894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d13ce-fc2a-4caf-a2cb-ff6c63dd169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.410008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 778/1548] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.410008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f441e2-7e40-45bf-921a-b8989f972f02 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.410353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-1b7d13ce-fc2a-4caf-a2cb-ff6c63dd169e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.410472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f441e2-7e40-45bf-921a-b8989f972f02 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.410472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f441e2-7e40-45bf-921a-b8989f972f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.410995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1b7d13ce-fc2a-4caf-a2cb-ff6c63dd169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.431124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c9f441e2-7e40-45bf-921a-b8989f972f02 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.432391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c9f441e2-7e40-45bf-921a-b8989f972f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.472157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 771/1549] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.476691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.477415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.477684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.496914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 779/1550] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.499710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.500957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.501110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1d19d-8e7b-4324-beb9-c4830b6a0198 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.501238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1d19d-8e7b-4324-beb9-c4830b6a0198 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.501555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1d19d-8e7b-4324-beb9-c4830b6a0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.509609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 772/1551] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.517727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.521939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-cfe1d19d-8e7b-4324-beb9-c4830b6a0198 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.522988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7356e-99a1-4a66-b395-2a391ccfd6d0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.523542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cfe1d19d-8e7b-4324-beb9-c4830b6a0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.527974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.528598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.528877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.529149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.529788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.531684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 780/1552] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.531813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.533553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40ed6-d350-4d04-af38-4834e156a197 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.533838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40ed6-d350-4d04-af38-4834e156a197 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.534115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e40ed6-d350-4d04-af38-4834e156a197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.554819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-56e40ed6-d350-4d04-af38-4834e156a197 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.556135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-56e40ed6-d350-4d04-af38-4834e156a197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.562318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 773/1553] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.563193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.564570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 781/1554] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.565816 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb4dc-49ab-49d2-b1b0-8dbd3bcc34d3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.566102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb4dc-49ab-49d2-b1b0-8dbd3bcc34d3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.566377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bb4dc-49ab-49d2-b1b0-8dbd3bcc34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.590089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ce6bb4dc-49ab-49d2-b1b0-8dbd3bcc34d3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.591673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce6bb4dc-49ab-49d2-b1b0-8dbd3bcc34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.633785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b697-57bd-4d8c-b940-4a7765801c10 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.638909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.639555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.639849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.640398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.640738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.641833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:52:54.641833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:52:54.641833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:52:54.643084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.655357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 782/1555] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.660673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b974c-bbee-4545-a33a-15874d5560f0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.661111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b974c-bbee-4545-a33a-15874d5560f0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.661383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b974c-bbee-4545-a33a-15874d5560f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.669530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 774/1556] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.673455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.683503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-158b974c-bbee-4545-a33a-15874d5560f0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.683804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7c11e-19d6-4437-95b5-6361ea91b4a4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.684731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-158b974c-bbee-4545-a33a-15874d5560f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.688336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.688957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.689227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.689494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.690144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.690446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.692091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf386531-1669-4f8e-adc4-80b7da4ad15e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.693287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 783/1557] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.695543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.695830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.696102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.705518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 775/1558] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.706795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.707070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.707367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.725242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.726480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.730013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.731115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.735067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 784/1559] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.735705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.739052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 776/1560] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.739908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.744672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:54.745924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d1f8-33ad-44cf-9a1c-05792efbd97c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.749898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d1b7-c002-419b-b180-77f47ec54da1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.751585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.752253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.752580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.752885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.753669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.755998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.758163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.758894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.759199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.759505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.760255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.760555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.762299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b073ebf-dd41-4613-9198-05efe6364ecf admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.776960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 777/1561] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.778399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.778684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.778973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.783288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:54.789724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 785/1562] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/users => generated 284 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.790713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.801640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eee7dc-37c6-4426-b363-c1d6657e641f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.804096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:54.805311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:54.807380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.808000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.808306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.808577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.809221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.810805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.813320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 778/1563] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:54.817300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.822450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 786/1564] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:54.824155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.824484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a848324-d62c-4e48-818c-fca5a274ee75 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.829429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.830058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.830326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.830588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.831262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.831561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.833185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.834935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a04a0-d38b-465b-a4e1-6f634bccb9ce admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.840454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.841090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.841367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.841658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.842302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.844318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.845690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 779/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.846610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.856665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d1db9-6164-4697-9895-3e4f583778f6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.861766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.862367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.862643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.862906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.863577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.864032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.865581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.875341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 787/1566] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.876219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.879561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 780/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.880471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.890999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03d69-5603-4c07-8904-e65bb621a384 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.946186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecb94-26a7-4d90-bd4a-7f9988a9bf86 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.951769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.952545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.952746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.952867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.953161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.953308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.953411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.953695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.953850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:54.954098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.954334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.955731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.956297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.970804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 788/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:54.971753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:54.983363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d643c21-9a5d-486d-8590-899f8b2be5ea admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:54.988390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:54.989012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:54.989285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:54.989548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:54.990189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:54.991883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:54.992575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 781/1569] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:54.993548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.003797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 789/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.004677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.015083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f99cdb-2a90-42aa-9a9b-2e7f9a1de1f4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.068719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de193ca-70c7-4c9b-a8d1-7f6169cb097e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.077544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.078569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.078997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.079205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.079509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.080187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.080583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:52:55.080710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.081058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.081173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.082092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.083871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.084807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.106022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 790/1571] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.106268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 782/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:52:55.106371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.106474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.119167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec1da8-7deb-4ed7-87ad-ebcbee7ae540 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.119566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f256b-f6a7-4bc6-b967-991b6fc8673e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.124586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.124796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.125214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.125358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.125492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.125747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.125843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.126105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.126383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.126987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.128398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.129168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.159024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 783/1573] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 297 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.160472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.164915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 791/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/projects => generated 357 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.166073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.172323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6e65-ede3-4c10-95e3-fd7bff5d28c1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.187640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:55.199241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:55.240468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e20531-7cdc-4c8a-854e-b6357fcb2aad admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.241018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.241674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.241952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.242213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.242861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.244849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.246096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.246785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.247114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.247465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.248153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.250124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.275477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 784/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.276499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.283145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 792/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.285503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.364887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e743c-919b-47d1-a30d-ee3aee247a71 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.371738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.372682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.373097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.373288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32678-47bf-4e04-a4ce-564a6c3581a6 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.373548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.374622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.378314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.381395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.382249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.382685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.383104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.384181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.386570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.402617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 793/1577] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.402617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.408289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 785/1578] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.408289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.418073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320568-8910-43af-ab72-fcddb0297c6a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.419770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bed6-b8f2-4ed3-a4d2-3832033781d6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.424052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.424867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.425098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.425343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.425645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.425817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] PATH_INFO: `/v3/projects/e775a06b78204f9192086fd27345738e/users/a68446ca08104be0bdbc41df89c8dc9a/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.426010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.426166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.426822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.429260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.444039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e775a06b78204f9192086fd27345738e, user_id=a68446ca08104be0bdbc41df89c8dc9a, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.445094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:52:55.445094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:52:55.445094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:52:55.447032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.461962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 786/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.462824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.464639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 794/1580] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:55 2022] PUT /identity/v3/projects/e775a06b78204f9192086fd27345738e/users/a68446ca08104be0bdbc41df89c8dc9a/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:55.466016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.546736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f853a-5b14-4587-aff8-39a08693d6d4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.549574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70f52-79fb-4579-af66-38467f1bea6c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.551667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.552306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.552630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.552959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.553592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.554506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.555119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.555423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.555583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.555717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.556325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.557983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.568798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 787/1581] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.572745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.586246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 795/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 279 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.586246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.586732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076623c0-128b-4865-b14c-0ebf2be12456 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.588181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.588807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.589078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.589353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.589985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.592560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.596974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46da5-7271-4893-8bf8-aecc71b31a47 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.600481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:55.606717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.607526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.607702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.607969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.608627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.610692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.622290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:55.624237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 788/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.625570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.639657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1d58-2214-450e-8e2c-49cf8b973e6c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.640088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 796/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 294 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.640986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.646996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.647874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.648270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.648593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] PATH_INFO: `/v3/projects/ab28c37dfab449dd9a3bbc5945dd7b55/users/2f302a1effab4b37ad99ca65f4559be9/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.653514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1183e8-089e-4f8c-aa69-4c99ee05df7b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.658670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.659313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.659589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.659858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.661074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.662153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.668455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab28c37dfab449dd9a3bbc5945dd7b55, user_id=2f302a1effab4b37ad99ca65f4559be9, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.671697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd8d8d-552f-4576-8363-54c50cda001b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.673997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 797/1585] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.676740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.687535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 789/1586] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:55 2022] PUT /identity/v3/projects/ab28c37dfab449dd9a3bbc5945dd7b55/users/2f302a1effab4b37ad99ca65f4559be9/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:55.688888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.689169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.689437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.691181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db8771-7e3b-4d38-89e9-335e94d6941a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.731964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] MFA Rules not processed for user `a68446ca08104be0bdbc41df89c8dc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:55.733305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:55.768135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.768756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.769032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.769297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.769931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.772045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.796056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 798/1587] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:55.800098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.810432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae24f1-9c7b-44cb-afa6-614fbaa5a884 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.813649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 790/1588] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:55.814808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.820033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.820661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.820933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.821196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.821845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.823488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.831821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865aff1e-6860-442a-9b37-49101cdf62b0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.834444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 799/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.835105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.844295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.845342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.845690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34858e-e9a0-42da-996c-eea05b181c9b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.845951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.846498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.847633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.850778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.851080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.851751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.852204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.852481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.853262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.854706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.867473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 800/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.868427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.869826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 791/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.871172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.885068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e333dd-0405-4d22-9a3f-8f0bcff228de admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.888097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0dc1f-0577-433d-8058-f40c58bc0a10 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.892955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.893939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.894396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.894818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.895185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.895931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.896169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.896600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.897061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.898083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.898781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4224b8c-c487-4e25-bb20-910e487ee52d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.900838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.915935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 801/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.915935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.916383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 792/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.916563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.916563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.916832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.928945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17102a-9807-40f7-a71c-8481d2c4e895 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.934408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.935133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.935501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.935856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:55.936573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:55.938261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:55.948373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] MFA Rules not processed for user `2f302a1effab4b37ad99ca65f4559be9`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:55.949974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:55.950137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 793/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:55.950833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:55.964360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc5d1-195e-4158-9ab0-04b52ca52694 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:55.974935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:55.976003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:55.976450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:55.976903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] PATH_INFO: `/v3/projects/6c458cf41ded4113a09fb07e10d81d39/users/76eb39c2010e43aaa0a01caaea3a38b9/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.013217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c458cf41ded4113a09fb07e10d81d39, user_id=76eb39c2010e43aaa0a01caaea3a38b9, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.018692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0fffa-0d37-4a94-8bdb-9f9e70ca9689 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.037684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 794/1595] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:55 2022] PUT /identity/v3/projects/6c458cf41ded4113a09fb07e10d81d39/users/76eb39c2010e43aaa0a01caaea3a38b9/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.039090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23196e-1f94-4c79-b196-8d029202689a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.039409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23196e-1f94-4c79-b196-8d029202689a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.039682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23196e-1f94-4c79-b196-8d029202689a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.041318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 795/1596] 10.222.0.26 () {54 vars in 962 bytes} [Mon Sep 12 23:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:52:56.042325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23196e-1f94-4c79-b196-8d029202689a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.046199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 802/1597] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:56.048965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.073967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5df0f-d629-4b78-a082-35d928b580f3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.106678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:56.111362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:56.117318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23196e-1f94-4c79-b196-8d029202689a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.124220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.124956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.125226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.125491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.126158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.127966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.146198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 796/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:56.146198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.150497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.151301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.151439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.151685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.152358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.154022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.154521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52e94-4268-490c-b9dc-01b725d6a393 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.160569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.161209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.161475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.161748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.162471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.164233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.168398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 803/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:56.169838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.176106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 797/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:56.177016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.187951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c0acc-86c2-44e9-aebc-f012c8cfe072 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.189802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3d73a-51de-4a40-a630-f6c56c8a08a8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.193339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.194141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.194519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.194889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] PATH_INFO: `/v3/projects/58a7b6a5dc86486682777285fa3295ac/users/d46cf7ae501740869d677d0e9e007041/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.194990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.195649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.195931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.196198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.196854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.198508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fd13a-df32-4f4b-a558-a20ff13cfd8d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.210597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 798/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:56.211911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.212192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.212464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.217135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a7b6a5dc86486682777285fa3295ac, user_id=d46cf7ae501740869d677d0e9e007041, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.219850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3aa04-d694-4c94-95d8-97957244a309 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.236568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:56.237393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 804/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:56 2022] PUT /identity/v3/projects/58a7b6a5dc86486682777285fa3295ac/users/d46cf7ae501740869d677d0e9e007041/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.237794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:56.238720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.239000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.239306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.281397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:56.283098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:56.311487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 799/1603] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:56.312277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.377218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 805/1604] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Sep 12 23:52:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:56.378485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.389756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707762d-a9fc-4591-84c7-bab64860c2cf None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.394059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.394695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.394967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.395305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.398715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343643c5-afc5-4632-b40a-a65a72868940 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.408498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.409190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.409454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.409727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] PATH_INFO: `/v3/projects/517176a7deb449fd8d17755dff6250d8/users/fe3f7fe0d41a4111b72fe0479fa2d55e/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.427193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=517176a7deb449fd8d17755dff6250d8, user_id=fe3f7fe0d41a4111b72fe0479fa2d55e, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.429988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.446109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 806/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:56 2022] PUT /identity/v3/projects/517176a7deb449fd8d17755dff6250d8/users/fe3f7fe0d41a4111b72fe0479fa2d55e/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.448914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.465136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.466974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.477377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 800/1606] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:56.478315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.491030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57ecd9-1d80-45e9-84b6-fa3e0431a6e0 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.520680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:56.523027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:56.528625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d08b2-81c0-40d3-89fe-1ee4b60b92bf admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.536472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.537400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.538155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.538492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] PATH_INFO: `/v3/projects/6c681f1232de4a5fbfc728c230f80624/users/a8cbfc74644f4d4594a70b833ad7002c/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.555076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c681f1232de4a5fbfc728c230f80624, user_id=a8cbfc74644f4d4594a70b833ad7002c, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.557887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.573573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 807/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:56 2022] PUT /identity/v3/projects/6c681f1232de4a5fbfc728c230f80624/users/a8cbfc74644f4d4594a70b833ad7002c/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.574510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.585969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.587065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.587881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.588284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] PATH_INFO: `/v3/projects/6b2f385a651c408bb9d32b5c4e1ab6ff/users/296f02c73fc548969bf60df4f8f372b0/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.617261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2f385a651c408bb9d32b5c4e1ab6ff, user_id=296f02c73fc548969bf60df4f8f372b0, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.622255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947b8ea-a6ad-42a0-a125-0c414222bb04 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.644345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 801/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:56 2022] PUT /identity/v3/projects/6b2f385a651c408bb9d32b5c4e1ab6ff/users/296f02c73fc548969bf60df4f8f372b0/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.652848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.653289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.654249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.683962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:56.686606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:56.690015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598cd68-e47f-4744-bd4c-d9e00cfb32e3 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.775184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.776663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.776894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.777186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 802/1609] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:56.777647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] PATH_INFO: `/v3/projects/f91493f2f0b34c5695131024db3f8baa/users/4ec1316c48f44a62af0cedf4adeadb60/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.778223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.804720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91493f2f0b34c5695131024db3f8baa, user_id=4ec1316c48f44a62af0cedf4adeadb60, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:56.808429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:56.836865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 808/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:56 2022] PUT /identity/v3/projects/f91493f2f0b34c5695131024db3f8baa/users/4ec1316c48f44a62af0cedf4adeadb60/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:56.837724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:56.867574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a9437-05f7-4545-b2ec-a44f551db273 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.925792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683985-e555-4ac0-a39f-f347a049d895 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:56.932519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.933164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.933444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.933717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.938722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:56.960909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:56.962368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:56.962824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:56.963308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:56.993001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:57.008732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.010176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a3b48-a0b1-4e96-90c2-6ff2a46ba77f service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.022582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 809/1611] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:57.024424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa5a7-cb18-40ab-898b-cb63d0d33d05 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.025309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa5a7-cb18-40ab-898b-cb63d0d33d05 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.025309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa5a7-cb18-40ab-898b-cb63d0d33d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.086812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-82aaa5a7-cb18-40ab-898b-cb63d0d33d05 None None] MFA Rules not processed for user `fe3f7fe0d41a4111b72fe0479fa2d55e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:57.087341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.089599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-82aaa5a7-cb18-40ab-898b-cb63d0d33d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:57.089998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c1ef2-0be0-4193-9b20-175b3c4649d0 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.108940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 803/1612] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:57.111257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6060f21-de48-4379-bdca-44fb2292c070 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.111746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6060f21-de48-4379-bdca-44fb2292c070 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.112485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6060f21-de48-4379-bdca-44fb2292c070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.153060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e6060f21-de48-4379-bdca-44fb2292c070 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:57.156536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e6060f21-de48-4379-bdca-44fb2292c070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:57.180915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 804/1613] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:57.188413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.189011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.189606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.256665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:57.259889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:57.302619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 805/1614] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:57.304676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.342078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac39e6-21b8-4b2e-9538-012c1c653ed3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.343200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 810/1615] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:57.345443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.348078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.349160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.367290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.368033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.368315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.368589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.391668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.396259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.404046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] MFA Rules not processed for user `4ec1316c48f44a62af0cedf4adeadb60`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:57.405645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:57.430438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 806/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:57.432330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.470835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae905b49-bcea-43e5-a880-281c41410c06 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.486393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.495667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.495667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.495667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.511433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.513808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.541229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 807/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:57.542121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.550994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b98df-df9e-4895-b1fb-a349c53aa60d service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.561332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.561847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.562135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.562410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.586517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.589118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.598204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 811/1618] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:57.598204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.612141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 808/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:57.613034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.629635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293f44-937a-4dcb-9e39-542e8a25f434 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.639858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.640442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.640710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2743a2-65fc-4a36-89c5-5573290cee94 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.640987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.641159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.641678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.643683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.646296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.647537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.648166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.649968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.654895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.664964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:57.700503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 809/1620] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:57.707792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.803621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 812/1621] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/projects => generated 357 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:57.810100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:57.822114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:57.927611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326040c-8ff4-4344-952d-b16e12cfeece admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.962234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f1cf3-250e-49dd-bef6-5834aacad2da admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:57.968010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:57.968903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:57.969189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:57.969479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:57.970166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:57.972324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34993a4-ef59-47ba-9953-67a41a030853 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.001998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 813/1622] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:52:58.003554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23af94e-2128-47c8-9809-c15138816087 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.003846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23af94e-2128-47c8-9809-c15138816087 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.007578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23af94e-2128-47c8-9809-c15138816087 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.009943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 814/1623] 10.222.0.26 () {54 vars in 962 bytes} [Mon Sep 12 23:52:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:52:58.010865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23af94e-2128-47c8-9809-c15138816087 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.027569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23af94e-2128-47c8-9809-c15138816087 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.036599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.037299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.037959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.038239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.038348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:58.039059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.040858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f345e4e-bfa9-49b5-b5e6-83bfe188e7ca admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.053114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 815/1624] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:58.059689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.059973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.060260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.060929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.062101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.062827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.063402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.064514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.068086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.096087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:58.097356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:58.107107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 810/1625] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:52:57 2022] POST /identity/v3/users => generated 293 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:52:58.107961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.120744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b6ef-1e4d-4d7d-87cf-5fbcfc275218 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.135524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.136542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.136863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.137204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] PATH_INFO: `/v3/projects/c4762704abee4fd095ece719ee0eab6c/users/05b1a38ac5e340d7aabd7d196c75abe3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.166197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4762704abee4fd095ece719ee0eab6c, user_id=05b1a38ac5e340d7aabd7d196c75abe3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.169668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.193042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 811/1626] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:58 2022] PUT /identity/v3/projects/c4762704abee4fd095ece719ee0eab6c/users/05b1a38ac5e340d7aabd7d196c75abe3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:58.194557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.212368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 816/1627] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Sep 12 23:52:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:58.213585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.232058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ba61a-d7cf-4f19-a8bc-53d5bad2be66 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.303986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17afdf1-be4d-4b09-a7a5-8140b3caabfc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.310404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.311429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.311790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.312660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.312992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.315094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1496073-09a2-4aa5-890e-61dd8558eac5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.330831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 812/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:58.334613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.335278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.335578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.336009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.336316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d342f7-8fac-4bba-9c3a-73b5b07cbf9d None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.336689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d342f7-8fac-4bba-9c3a-73b5b07cbf9d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.337060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d342f7-8fac-4bba-9c3a-73b5b07cbf9d None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.337180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.338888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 813/1629] 10.222.0.26 () {54 vars in 912 bytes} [Mon Sep 12 23:52:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:52:58.340141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d342f7-8fac-4bba-9c3a-73b5b07cbf9d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.341445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a87d4-934a-49b1-98ab-d16d6871653a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.358736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d342f7-8fac-4bba-9c3a-73b5b07cbf9d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.361350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 817/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:58.363864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.364402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.364678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.368176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.368902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.369189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.369480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] PATH_INFO: `/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a/users/d3ac2cbcbdb64c2ca9befc94d15c7bad/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.388199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:58.389465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:58.391590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc25286b0ed4dbdbc63be310ec92d6a, user_id=d3ac2cbcbdb64c2ca9befc94d15c7bad, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.394836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:58.397078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.423097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 814/1631] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:58 2022] PUT /identity/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a/users/d3ac2cbcbdb64c2ca9befc94d15c7bad/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:58.423749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.503135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 818/1632] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:52:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:58.505037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.524152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6055faa-b04c-433d-8c2b-1263a318570b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.541324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbbede-059d-450b-a163-77ee573ef65d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.547280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.548481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.548481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.548481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] PATH_INFO: `/v3/projects/c4762704abee4fd095ece719ee0eab6c/users/05b1a38ac5e340d7aabd7d196c75abe3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.550599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4762704abee4fd095ece719ee0eab6c, user_id=05b1a38ac5e340d7aabd7d196c75abe3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.553412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.577227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 815/1633] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:58 2022] PUT /identity/v3/projects/c4762704abee4fd095ece719ee0eab6c/users/05b1a38ac5e340d7aabd7d196c75abe3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:58.577227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.622711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:58.630013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.631050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.632100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.632423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.633646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.636251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd1ad-8dc7-4a8f-ac40-68422ae8015f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.650920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 819/1634] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:52:58.652486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaebca-c489-48e9-acd4-27b4d4b8edd7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.652853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaebca-c489-48e9-acd4-27b4d4b8edd7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.653217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaebca-c489-48e9-acd4-27b4d4b8edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.683606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cb726-4f88-40b5-b137-abd82eb74813 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.688072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.688734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.689077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.689365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.717135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0ceaebca-c489-48e9-acd4-27b4d4b8edd7 None None] MFA Rules not processed for user `05b1a38ac5e340d7aabd7d196c75abe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:58.720015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0ceaebca-c489-48e9-acd4-27b4d4b8edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:58.812388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.817123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:58.844160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 816/1635] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:58.845163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.879495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 820/1636] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:58.883687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d769-a7fb-4eee-b0d1-1d59bd0582af None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.884547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d769-a7fb-4eee-b0d1-1d59bd0582af None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.885729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d769-a7fb-4eee-b0d1-1d59bd0582af None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.890105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 821/1637] 10.222.0.26 () {54 vars in 912 bytes} [Mon Sep 12 23:52:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:52:58.892175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8d769-a7fb-4eee-b0d1-1d59bd0582af None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:58.916771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8d769-a7fb-4eee-b0d1-1d59bd0582af None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:58.924777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:58.926018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:58.926513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:58.927007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:58.973260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:58.978537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede989a-b2fd-460c-ad3f-71ba0023bf3c service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:59.006282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392591-6727-471c-96d8-c32e21be3c30 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:59.022772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:59.023681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.024030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.024376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] PATH_INFO: `/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a/users/d3ac2cbcbdb64c2ca9befc94d15c7bad/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.027705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc25286b0ed4dbdbc63be310ec92d6a, user_id=d3ac2cbcbdb64c2ca9befc94d15c7bad, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:59.031558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418eb3a-2372-4a28-87bd-987d76d86252 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:59.037483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 822/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:59.044174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.045994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.046516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.054278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 817/1639] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:52:58 2022] PUT /identity/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a/users/d3ac2cbcbdb64c2ca9befc94d15c7bad/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:59.076367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.077704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.080527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.122303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:59.124085 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:59.134326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:59.155121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] MFA Rules not processed for user `d3ac2cbcbdb64c2ca9befc94d15c7bad`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:59.157853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:59.339562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:52:59.360494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 823/1640] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:52:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:59.360494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:59.427331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 818/1641] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:52:59.428522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:59.514068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d53d4-a207-4846-af93-86e826c9dbc7 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:59.521918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:59.522992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.523500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.523974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.539953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7974aaa-582b-4564-a39e-1c6eb41b7981 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:59.573059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:59.576969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.580904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.583140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] PATH_INFO: `/v3/users/a68446ca08104be0bdbc41df89c8dc9a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.602853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a68446ca08104be0bdbc41df89c8dc9a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:59.607522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:59.674019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:59.677659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:59.713432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 824/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:59.715087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:59.733357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a3e1e-7496-49ef-8b0b-4d553986bcf3 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:52:59.742017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:52:59.742939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.743264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.743751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.772897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:52:59.775891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893003d1-b553-454b-a68c-25da0212b818 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:52:59.776934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a68446ca08104be0bdbc41df89c8dc9a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:52:59.778123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a68446ca08104be0bdbc41df89c8dc9a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:52:59.779021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a68446ca08104be0bdbc41df89c8dc9a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:52:59.795903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a68446ca08104be0bdbc41df89c8dc9a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:52:59.811054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 825/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:52:59.824490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a68446ca08104be0bdbc41df89c8dc9a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:52:59.868025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 819/1644] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:52:59 2022] DELETE /identity/v3/users/a68446ca08104be0bdbc41df89c8dc9a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:52:59.874003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854049-ed06-455c-9472-929d52039708 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:52:59.874642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854049-ed06-455c-9472-929d52039708 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:52:59.875208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71854049-ed06-455c-9472-929d52039708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:52:59.920211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-71854049-ed06-455c-9472-929d52039708 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:52:59.923997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-71854049-ed06-455c-9472-929d52039708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:52:59.988460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:52:59.992411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:00.045263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:00.115990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 826/1645] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:00.154095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c8712-b6e0-4263-add3-a36802f6ef9a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:00.163215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:00.164264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:00.164453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:00.166621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:00.171740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:00.191705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854049-ed06-455c-9472-929d52039708 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:00.193782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-71854049-ed06-455c-9472-929d52039708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:00.220090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:00.360262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71854049-ed06-455c-9472-929d52039708 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:00.366701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:00.367478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:00.368104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:00.381931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:00.408833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:00.413449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:00.432228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:00.435726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780560f-faaa-4982-a232-37cd3234e326 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:00.444620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 827/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:00.476108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 820/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:52:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:01.901764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:01.905402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:01.939821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27b99-42bb-4618-8fa7-516a7d293049 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:01.949022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:01.950462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:01.950462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:01.950462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] PATH_INFO: `/v3/users/4ec1316c48f44a62af0cedf4adeadb60` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:01.951695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec1316c48f44a62af0cedf4adeadb60)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:01.955290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: WARNING py.warnings [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:53:01.955290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: warnings.warn(msg) Sep 12 23:53:01.955290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: Sep 12 23:53:01.957550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:02.197721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec1316c48f44a62af0cedf4adeadb60'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.200843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec1316c48f44a62af0cedf4adeadb60'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.232910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec1316c48f44a62af0cedf4adeadb60'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.267616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec1316c48f44a62af0cedf4adeadb60'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.354019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec1316c48f44a62af0cedf4adeadb60'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.378154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 828/1648] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:01 2022] DELETE /identity/v3/users/4ec1316c48f44a62af0cedf4adeadb60 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:02.381234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21d4b-d014-4acc-9522-cb9fd406ae7c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:02.381726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21d4b-d014-4acc-9522-cb9fd406ae7c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:02.382077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c21d4b-d014-4acc-9522-cb9fd406ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:02.389549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:02.404589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:02.487214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-52c21d4b-d014-4acc-9522-cb9fd406ae7c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:02.494519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-52c21d4b-d014-4acc-9522-cb9fd406ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:02.702755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:02.716146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b638-a380-4d45-bd99-05e0faafcab8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:02.725419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:02.726558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:02.726936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:02.727372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] PATH_INFO: `/v3/projects/e775a06b78204f9192086fd27345738e` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:02.729993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e775a06b78204f9192086fd27345738e)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:02.732194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:02.747877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 821/1649] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:02.838871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e775a06b78204f9192086fd27345738e', 'request_id': 'req-54d8be67-770f-4b8d-8af2-515f25bff182'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.852178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e775a06b78204f9192086fd27345738e', 'request_id': 'req-54d8be67-770f-4b8d-8af2-515f25bff182'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.870391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e775a06b78204f9192086fd27345738e', 'request_id': 'req-54d8be67-770f-4b8d-8af2-515f25bff182'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.900213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] The token cache is being invalidate because project e775a06b78204f9192086fd27345738e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:02.900752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.901825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:02.904475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 829/1650] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:02 2022] DELETE /identity/v3/projects/e775a06b78204f9192086fd27345738e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:02.911267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75330722-69b2-44dc-b7de-acc7d19c755f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:02.911876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75330722-69b2-44dc-b7de-acc7d19c755f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:02.912467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75330722-69b2-44dc-b7de-acc7d19c755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:02.916757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:02.918463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:02.987355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-75330722-69b2-44dc-b7de-acc7d19c755f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:02.990295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-75330722-69b2-44dc-b7de-acc7d19c755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:03.076548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8be67-770f-4b8d-8af2-515f25bff182 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.087196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.088014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.088392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.088728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.094595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:03.217094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 822/1651] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:03.219496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.220004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.220495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.254583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.257872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:03.261395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:03.262567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:03.280696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 823/1652] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:03.282602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:03.288379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 830/1653] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:03.299010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1211b-ed43-499a-a7b7-c63c8caacd04 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.310435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.313829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.313829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.314200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.315344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:03.315751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.318392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:03.344406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 824/1654] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:03.344712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:03.360436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ab003-e423-4ea6-9cba-33833f676f77 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.378339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.380126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.381574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.382121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.384146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.390267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:03.484205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 831/1655] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:03 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:03.487509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:03.505490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82b7f5-91bc-4ce3-bf7d-292d0b19bbaf admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.649005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:03.784018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.785625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.785625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.785625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.786379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.792987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:03.851986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 825/1656] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:03 2022] POST /identity/v3/users => generated 287 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:03.852516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:03.885172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0050d8-9af3-4a67-aa65-8c2b0815b944 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.901767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.903164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.914766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:03.933652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 832/1657] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:03.938057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:03.960755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0a52e-985f-4f1b-8074-d26d4d421b22 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:03.974095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:03.975405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:03.975405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:03.975405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:03.976046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:03.977777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef42325-8b59-43e0-8829-0d88779a2d80 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:04.003976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 826/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:04.004455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:04.037488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:04.045615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a324-81e2-4887-b68d-d88ffbcd72cd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:04.058684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:04.061163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:04.062197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:04.062973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] PATH_INFO: `/v3/projects/33db7a46234c423fb07ab2111c9460df/users/0c9b4c6e52724f498513a888884a6a9c/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:04.139822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33db7a46234c423fb07ab2111c9460df, user_id=0c9b4c6e52724f498513a888884a6a9c, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:04.146566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:04.186345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 833/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:04 2022] PUT /identity/v3/projects/33db7a46234c423fb07ab2111c9460df/users/0c9b4c6e52724f498513a888884a6a9c/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:04.198098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:04.198559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:04.198725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:04.264388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] MFA Rules not processed for user `0c9b4c6e52724f498513a888884a6a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:04.266168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:04.487542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:04.490296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 827/1660] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:04.617630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771dbe-0e60-417c-a129-a1592b221a80 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:04.624097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:04.624745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:04.625102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:04.625735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:04.626703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:04.631841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:04.708028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 834/1661] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:04 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:04.713794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:04.736707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4cc6f-6560-4537-8a6a-819e0233c356 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:04.928891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:04.929953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:04.930429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:04.930905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:04.932122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:04.935903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.000906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 828/1662] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:04 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:05.007185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.053127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1865f-c6dc-4141-8f87-2899610f4477 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:05.078401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:05.079637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.080178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.080854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.082356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:05.085885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.143631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 835/1663] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:05.151331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.196603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cabdd-f8dd-459d-8867-3a5cfab640c6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:05.210400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:05.211255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.211537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.211819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] PATH_INFO: `/v3/projects/14e6cff0cad2468c81801a099f74b950/users/11fda0a40de24305bc18a12bb4454288/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.264236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e6cff0cad2468c81801a099f74b950, user_id=11fda0a40de24305bc18a12bb4454288, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:05.276304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.310099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 829/1664] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:05 2022] PUT /identity/v3/projects/14e6cff0cad2468c81801a099f74b950/users/11fda0a40de24305bc18a12bb4454288/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:05.321309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.335972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270424-3a52-4bb4-8c60-1635d610df58 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:05.544679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:05.545385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.545695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.545968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.546645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:05.548378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.574255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 836/1665] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:05.577574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.594205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:05.597153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.597541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:05.602511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914af4-76cc-4450-b430-d2efe418f94b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:05.621313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:05.623297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.623297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.623297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] PATH_INFO: `/v3/projects/14e6cff0cad2468c81801a099f74b950/users/11fda0a40de24305bc18a12bb4454288/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.636267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a2de-fc93-4ea4-a286-d2c4a79934ac admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:05.655122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:05.659189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.659499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.659779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] PATH_INFO: `/v3/projects/f91493f2f0b34c5695131024db3f8baa` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.661298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e6cff0cad2468c81801a099f74b950, user_id=11fda0a40de24305bc18a12bb4454288, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:05.662462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91493f2f0b34c5695131024db3f8baa)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:05.671543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b9899-6a27-47ab-8486-8f3c4a9585ab admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.675325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:05.727527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 830/1666] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:05 2022] PUT /identity/v3/projects/14e6cff0cad2468c81801a099f74b950/users/11fda0a40de24305bc18a12bb4454288/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:05.739077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207e0a-4297-437b-987d-03903326ee39 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:05.739077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207e0a-4297-437b-987d-03903326ee39 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:05.739504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54207e0a-4297-437b-987d-03903326ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:05.818033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:05.824155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91493f2f0b34c5695131024db3f8baa', 'request_id': 'req-f3d1018f-48be-4137-a276-443996b93654'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:05.862395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91493f2f0b34c5695131024db3f8baa', 'request_id': 'req-f3d1018f-48be-4137-a276-443996b93654'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:05.878470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91493f2f0b34c5695131024db3f8baa', 'request_id': 'req-f3d1018f-48be-4137-a276-443996b93654'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:05.882793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-54207e0a-4297-437b-987d-03903326ee39 None None] MFA Rules not processed for user `11fda0a40de24305bc18a12bb4454288`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:05.885346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-54207e0a-4297-437b-987d-03903326ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:05.910031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] The token cache is being invalidate because project f91493f2f0b34c5695131024db3f8baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:05.910930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:05.912149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:05.916004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 837/1667] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:05 2022] DELETE /identity/v3/projects/f91493f2f0b34c5695131024db3f8baa => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:05.921689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:05.924482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:06.142134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 831/1668] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:06.154153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1018f-48be-4137-a276-443996b93654 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:06.185004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:06.185769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:06.186191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:06.186353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] PATH_INFO: `/v3/users/d3ac2cbcbdb64c2ca9befc94d15c7bad` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:06.194913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ac2cbcbdb64c2ca9befc94d15c7bad)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:06.203754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:06.273460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207e0a-4297-437b-987d-03903326ee39 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:06.275136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-54207e0a-4297-437b-987d-03903326ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:06.362032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ac2cbcbdb64c2ca9befc94d15c7bad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:06.362819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ac2cbcbdb64c2ca9befc94d15c7bad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:06.384667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ac2cbcbdb64c2ca9befc94d15c7bad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:06.420227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ac2cbcbdb64c2ca9befc94d15c7bad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:06.444567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ac2cbcbdb64c2ca9befc94d15c7bad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:06.460248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 838/1669] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:05 2022] DELETE /identity/v3/users/d3ac2cbcbdb64c2ca9befc94d15c7bad => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:06.546183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54207e0a-4297-437b-987d-03903326ee39 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:06.878053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:06.879455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:06.880095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:06.882619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:06.902029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:07.058802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:07.108922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:07.110607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:07.146981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 832/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 874 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:07.451200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:07.453711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:07.505744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7c13-7c89-4777-b452-d4ee6d1d128a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:07.519456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:07.524041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:07.524544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:07.525225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:07.535249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:07.797905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:07.849513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:07.856370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d897e-b3db-4d34-8721-668a3b12c4df service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:07.904813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 839/1671] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:09.399770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:09.402204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:09.613282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadeac1-9d33-4f29-bc77-43288186a4e6 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:09.640924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:09.646915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:09.649002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:09.649364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] PATH_INFO: `/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:09.654282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc25286b0ed4dbdbc63be310ec92d6a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:09.658167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: WARNING py.warnings [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 12 23:53:09.658167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: warnings.warn(msg) Sep 12 23:53:09.658167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: Sep 12 23:53:09.661903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:09.827741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc25286b0ed4dbdbc63be310ec92d6a', 'request_id': 'req-cf10e854-2235-4ee2-a112-215e21c0896a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:09.896006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc25286b0ed4dbdbc63be310ec92d6a', 'request_id': 'req-cf10e854-2235-4ee2-a112-215e21c0896a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:09.932579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc25286b0ed4dbdbc63be310ec92d6a', 'request_id': 'req-cf10e854-2235-4ee2-a112-215e21c0896a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:09.995283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] The token cache is being invalidate because project acc25286b0ed4dbdbc63be310ec92d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:09.995804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:10.003951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-cf10e854-2235-4ee2-a112-215e21c0896a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:10.020354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 833/1672] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:09 2022] DELETE /identity/v3/projects/acc25286b0ed4dbdbc63be310ec92d6a => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:10.025602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:10.026173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:10.026344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:10.096878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:10.099515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:10.359840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 840/1673] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:10.380683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:10.380959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:10.381229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:10.428902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:10.435716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:10.471696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 834/1674] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:10.478187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:10.491751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a6c35-434e-442c-98ac-b7b8a6b8c404 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:10.529739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:10.530102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:10.530384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:10.530662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:10.531377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:10.532058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:10.533752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:10.595691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 841/1675] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:10.603345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:10.620654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1bdf1-e9ef-4358-9247-650e672ed7ec None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:10.639058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:10.642021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:10.642302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:10.646572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:10.647578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:10.651699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:10.787697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 835/1676] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:10 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:10.795391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:10.845840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3aff-b88b-476c-b487-d1b3be7f23d1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:11.135168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:11.147792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.158042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.158370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.159114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:11.161245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:11.242079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:11.243100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 842/1677] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:10 2022] POST /identity/v3/users => generated 283 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:11.258562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d3fb5-d25b-4be8-860c-80e57c484d2d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:11.269579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:11.272336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:11.301483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:11.302891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.303114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.303509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.305733 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:11.308446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:11.322764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba29a7-8f36-4c75-aaac-118535b3a940 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:11.346557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 836/1678] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:11.356402 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:11.358204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:11.358851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.361081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.361656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] PATH_INFO: `/v3/users/2f302a1effab4b37ad99ca65f4559be9` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.369746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f302a1effab4b37ad99ca65f4559be9)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:11.372182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6fc8f-da14-46e9-86b0-3c75d09df708 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:11.374895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:11.389709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:11.390993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.391309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.391582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.392269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:11.393952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:11.429010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 837/1679] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:11.432463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:11.459710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03100-ab3b-4529-bb0b-9851da21910d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:11.477510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:11.480115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.481383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.482118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] PATH_INFO: `/v3/projects/15ce950d29b644bd84c483561209ce5d/users/da0ffbdb13db47b9a95cffc2f8cbe51c/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.487787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f302a1effab4b37ad99ca65f4559be9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:11.488768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f302a1effab4b37ad99ca65f4559be9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:11.506500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f302a1effab4b37ad99ca65f4559be9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:11.523429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f302a1effab4b37ad99ca65f4559be9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:11.527184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15ce950d29b644bd84c483561209ce5d, user_id=da0ffbdb13db47b9a95cffc2f8cbe51c, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:11.528356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:11.534106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798492c-4b50-453d-b6a0-b1561d21e3e6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:11.557313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:11.566172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-456be75c-6de7-4c30-bf37-0088be566262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f302a1effab4b37ad99ca65f4559be9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:11.584978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 838/1680] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:11 2022] PUT /identity/v3/projects/15ce950d29b644bd84c483561209ce5d/users/da0ffbdb13db47b9a95cffc2f8cbe51c/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:11.589231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.589847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.590257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.596244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 843/1681] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:11 2022] DELETE /identity/v3/users/2f302a1effab4b37ad99ca65f4559be9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:11.609392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:11.609392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:11.609392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:11.661017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:11.674963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:11.739835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] MFA Rules not processed for user `da0ffbdb13db47b9a95cffc2f8cbe51c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:11.750470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:12.056872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 839/1682] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:12.060827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:12.070483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:12.081311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 844/1683] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:12.120301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5fe41-f8f1-441d-b83a-2b2020be2dea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:12.147611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:12.149213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:12.150052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:12.150770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:12.152615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:12.161893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:12.244662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:12.247998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:12.260099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 840/1684] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:12 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:12.261719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:12.588908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6b1aa-5276-41ca-adab-055241ad5b1d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:12.671852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38709690-07af-4d21-9d30-3452cc1e63de admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:12.710399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:12.712240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:12.715895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:12.716748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:12.721066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:12.730818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:12.801493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 841/1685] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:12 2022] POST /identity/v3/users => generated 288 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:12.809246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:12.837314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca13c-6ad5-4b5a-9cfc-0dbb69cdd4dc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:12.850700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:12.852966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:12.854625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:12.855164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:12.855537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:12.856726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:12.857341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:12.857768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:12.858205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:12.863518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:12.902986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 842/1686] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:12.906795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:12.945190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:12.948146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:12.951356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2970c-4eb8-46d3-8a1f-eae7f96f0afe admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:12.965190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:12.968021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:12.968804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:12.969120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] PATH_INFO: `/v3/projects/4d827105b9bd47a0bf020c56e7480d02/users/839a3d16c0e143b3b62846ae297ffdf5/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:13.013484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d827105b9bd47a0bf020c56e7480d02, user_id=839a3d16c0e143b3b62846ae297ffdf5, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:13.017230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:13.040644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 845/1687] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 798 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:13.075535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 843/1688] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:12 2022] PUT /identity/v3/projects/4d827105b9bd47a0bf020c56e7480d02/users/839a3d16c0e143b3b62846ae297ffdf5/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:13.080627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:13.124826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8eab9-f448-4915-9f8c-ed5b3f4b51f8 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:13.290263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:13.376233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:13.377184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:13.377729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:13.378209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:13.380948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:13.388728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b17519-ea09-416b-baf7-b4898432f6a5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:13.439451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 846/1689] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:13.442851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:13.482177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63d068-0a15-429d-918b-afc09146bbd5 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:13.517016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:13.520887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:13.521862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:13.524008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] PATH_INFO: `/v3/projects/4d827105b9bd47a0bf020c56e7480d02/users/839a3d16c0e143b3b62846ae297ffdf5/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:13.536293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d827105b9bd47a0bf020c56e7480d02, user_id=839a3d16c0e143b3b62846ae297ffdf5, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:13.552043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:13.598870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 844/1690] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:13 2022] PUT /identity/v3/projects/4d827105b9bd47a0bf020c56e7480d02/users/839a3d16c0e143b3b62846ae297ffdf5/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:13.624197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:13.624750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:13.625863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:13.733719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] MFA Rules not processed for user `839a3d16c0e143b3b62846ae297ffdf5`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:13.737062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:13.912701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 847/1691] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:13.931033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:13.933775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:13.948346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:14.088883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a6522-1238-457c-836f-d572a340aaba admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:14.093422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:14.094081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:14.096053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:14.096456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:14.109605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:14.111179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:14.153012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 845/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:14.472780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:14.474488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:14.521128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4992-4009-482c-b814-58dee899fbb6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:14.531805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:14.536119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:14.536651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:14.537214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:14.757885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:14.759322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee174-9c0f-4cda-be3e-f856509b3b77 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:14.782314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 848/1693] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:15.638032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:15.639535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:15.970301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d236f-95ea-45b7-89e6-bdde4360b6ec service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:15.990732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:15.991875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:15.992346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:15.992817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] PATH_INFO: `/v3/projects/ab28c37dfab449dd9a3bbc5945dd7b55` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.001008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab28c37dfab449dd9a3bbc5945dd7b55)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:16.012457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:16.012587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07cab-f3fb-4f13-aba4-cdb8407b25cb None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.013061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07cab-f3fb-4f13-aba4-cdb8407b25cb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.014594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07cab-f3fb-4f13-aba4-cdb8407b25cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.154521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab28c37dfab449dd9a3bbc5945dd7b55', 'request_id': 'req-8740b970-04f4-4581-81d9-ff091672e7e0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:16.165991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab28c37dfab449dd9a3bbc5945dd7b55', 'request_id': 'req-8740b970-04f4-4581-81d9-ff091672e7e0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:16.171116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a3f07cab-f3fb-4f13-aba4-cdb8407b25cb None None] MFA Rules not processed for user `d46cf7ae501740869d677d0e9e007041`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:16.175403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a3f07cab-f3fb-4f13-aba4-cdb8407b25cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:16.188678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab28c37dfab449dd9a3bbc5945dd7b55', 'request_id': 'req-8740b970-04f4-4581-81d9-ff091672e7e0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:16.211917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] The token cache is being invalidate because project ab28c37dfab449dd9a3bbc5945dd7b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:16.212361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:16.213004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8740b970-04f4-4581-81d9-ff091672e7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:16.218092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 846/1694] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:15 2022] DELETE /identity/v3/projects/ab28c37dfab449dd9a3bbc5945dd7b55 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:16.219743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25583-100f-4607-a6a4-20cc9a4aa950 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.220039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25583-100f-4607-a6a4-20cc9a4aa950 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.220378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a25583-100f-4607-a6a4-20cc9a4aa950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.260724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-03a25583-100f-4607-a6a4-20cc9a4aa950 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:16.264006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-03a25583-100f-4607-a6a4-20cc9a4aa950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:16.353147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:16.362906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:16.411422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 849/1695] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:16.423561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.423961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.424540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.431540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 847/1696] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:16.442263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.442544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.442835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.487745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] MFA Rules not processed for user `d46cf7ae501740869d677d0e9e007041`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:16.489370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:16.499828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:16.501117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:16.508063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 850/1697] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:16.521039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:16.522611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:16.531891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 848/1698] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:16.534973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:16.561067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d4cd-de3b-42c5-82d0-6869e51e8bb2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:16.601282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:16.602259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.602739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.603182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.604371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:16.604907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:16.607993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:16.678194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 849/1699] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:16.679672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:16.720118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7d2d6-05eb-40b8-948d-066242424f41 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:16.739053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a744600-46bf-402c-8f5a-77cd8f80493a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:16.740093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:16.741135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.741585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.742031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.743170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:16.746862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:16.756038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:16.758207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:16.758759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:16.759485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:16.804577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:16.806488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:16.847260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 851/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:16.903610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:16.903771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 850/1701] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/projects => generated 405 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:16.949683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35962334-8c0b-4de0-ab8c-081efadfbaeb service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:17.204632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:17.205619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:17.205954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:17.206235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:17.206929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:17.209196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:17.309699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 852/1702] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:16 2022] POST /identity/v3/users => generated 303 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:17.314148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:17.328929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0030e2-3291-4555-8c84-b96234f78b77 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:17.357280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:17.357936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:17.358254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:17.358531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:17.359153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:17.360897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:17.401300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 851/1703] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:17.405724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:17.426452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a88a-235d-4b78-8b99-fb1e15417de4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:17.448594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:17.449267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:17.449549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:17.449828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:17.450503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:17.452247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ae2d-271d-4bd0-a6e9-dc7460b2d0f3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:17.482008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 853/1704] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:17.486692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:17.505108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2021-e493-4b06-91e3-c32aad566c9e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:17.519035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:17.520022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:17.520448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:17.520864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] PATH_INFO: `/v3/projects/0f1fcf9b05564a3487c7e85986d07567/users/9f25b2bbb8d54599b1cf8a9f632d10aa/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:17.560858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1fcf9b05564a3487c7e85986d07567, user_id=9f25b2bbb8d54599b1cf8a9f632d10aa, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:17.564152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:17.599727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 852/1705] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:17 2022] PUT /identity/v3/projects/0f1fcf9b05564a3487c7e85986d07567/users/9f25b2bbb8d54599b1cf8a9f632d10aa/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:17.611623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2f43-6374-41a1-af12-3901ca84f609 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:17.611908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2f43-6374-41a1-af12-3901ca84f609 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:17.612189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2f43-6374-41a1-af12-3901ca84f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:17.676887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d5ab2f43-6374-41a1-af12-3901ca84f609 None None] MFA Rules not processed for user `9f25b2bbb8d54599b1cf8a9f632d10aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:17.679068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d5ab2f43-6374-41a1-af12-3901ca84f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:17.841053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:18.033621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 854/1706] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:18.039269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:18.123949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a5db-5ff9-487a-8738-75d7e7ae993c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:18.124488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a5db-5ff9-487a-8738-75d7e7ae993c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:18.126417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a5db-5ff9-487a-8738-75d7e7ae993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:18.293003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-64e6a5db-5ff9-487a-8738-75d7e7ae993c None None] MFA Rules not processed for user `76eb39c2010e43aaa0a01caaea3a38b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:18.302833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-64e6a5db-5ff9-487a-8738-75d7e7ae993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:18.340176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5e63-cc6b-4824-bac6-3958624fa3db admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:18.381802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:18.385294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:18.385710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:18.386151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:18.387322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:18.402905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a6333-0318-41ff-a980-8ecc0b78c278 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:18.540316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 853/1707] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:18 2022] POST /identity/v3/projects => generated 405 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:18.553564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f04a7-880e-4407-add1-dd0844c86ea3 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:18.554053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f04a7-880e-4407-add1-dd0844c86ea3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:18.554550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f04a7-880e-4407-add1-dd0844c86ea3 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:18.557071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 854/1708] 10.222.0.26 () {54 vars in 912 bytes} [Mon Sep 12 23:53:18 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:53:18.564352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f04a7-880e-4407-add1-dd0844c86ea3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:18.622259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 855/1709] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:18.624109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f0256-8be2-4980-9c68-e4e094d1e99a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:18.630883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f0256-8be2-4980-9c68-e4e094d1e99a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:18.631767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f0256-8be2-4980-9c68-e4e094d1e99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:18.761028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f39f0256-8be2-4980-9c68-e4e094d1e99a None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:18.766342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f39f0256-8be2-4980-9c68-e4e094d1e99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:18.923709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:18.946396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 856/1710] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:53:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:18.948574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:18.948856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:18.949124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.007091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] MFA Rules not processed for user `76eb39c2010e43aaa0a01caaea3a38b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:19.010142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:19.024860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f04a7-880e-4407-add1-dd0844c86ea3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:19.048072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:19.050176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:19.051065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:19.051485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.052619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:19.074317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:19.197166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 855/1711] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:18 2022] POST /identity/v3/users => generated 309 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:19.199324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:19.213215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:19.279464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef1251-4966-4de7-b9ac-476582a3b4e0 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:19.285869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 857/1712] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:19.288320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:19.317624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:19.326256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ba5f-2ddf-413b-b508-5f51da8df115 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:19.326792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:19.326792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:19.327097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.347366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:19.348388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:19.348717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:19.348950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.350691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:19.357161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:19.359566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:19.374724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:19.383524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 858/1713] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:19.385009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:19.386345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:19.387674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:19.430892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 856/1714] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:19.436824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:19.541528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de9f18-8d64-4329-954b-01ad0e09cda4 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:19.588551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:19.590105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:19.590770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:19.591593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] PATH_INFO: `/v3/projects/26f349c524f24c848a43920142474e53/users/ca962d04ed2b4208a0157ae4b04faf7c/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.700703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279aad6-0779-4227-85a6-1c0f6cea0d23 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:19.712890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f349c524f24c848a43920142474e53, user_id=ca962d04ed2b4208a0157ae4b04faf7c, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:19.732712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:19.733377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:19.733689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:19.733968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:19.741161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:19.779171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:19.783245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:19.791315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 857/1715] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:19 2022] PUT /identity/v3/projects/26f349c524f24c848a43920142474e53/users/ca962d04ed2b4208a0157ae4b04faf7c/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:19.797832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:19.826815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 859/1716] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:20.212245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5c95e-c472-4c0e-90a5-ae8e720839ec admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:20.233033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:20.236511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:20.238844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:20.239697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:20.244935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:20.255031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:20.354563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:20.356197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 858/1717] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 558 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:20.387590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a10837-0e13-4fc9-afc2-4753a384cf26 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:20.389092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:20.393034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:20.394097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:20.394863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:20.395263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:20.395605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] PATH_INFO: `/v3/projects/26f349c524f24c848a43920142474e53/users/ca962d04ed2b4208a0157ae4b04faf7c/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:20.398311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f349c524f24c848a43920142474e53, user_id=ca962d04ed2b4208a0157ae4b04faf7c, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:20.401640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7c7de-5120-4695-a47a-61509041976f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:20.427277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 860/1718] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:20 2022] PUT /identity/v3/projects/26f349c524f24c848a43920142474e53/users/ca962d04ed2b4208a0157ae4b04faf7c/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:20.442465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:20.443316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:20.443803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:20.453008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d106-bde7-4a17-9e49-7fad4e13302d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:20.516337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] MFA Rules not processed for user `ca962d04ed2b4208a0157ae4b04faf7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:20.518926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:20.828719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 861/1719] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:20 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:20.828896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:20.830556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:20.831178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:20.860150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] PATH_INFO: `/v3/users/da0ffbdb13db47b9a95cffc2f8cbe51c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:20.866950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0ffbdb13db47b9a95cffc2f8cbe51c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:20.877413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:20.913812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:20.915784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:21.084896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca2c69-3852-4572-a728-9b2210d291f8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:21.103036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:21.103777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:21.104062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:21.104322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:21.125033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:21.128672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:21.134567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:21.135464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8c54-6d79-4c4b-8cb0-ce7e603390e6 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:21.167871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0ffbdb13db47b9a95cffc2f8cbe51c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:21.171263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0ffbdb13db47b9a95cffc2f8cbe51c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:21.176901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 862/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:21.212041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0ffbdb13db47b9a95cffc2f8cbe51c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:21.249477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0ffbdb13db47b9a95cffc2f8cbe51c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:21.341290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0ffbdb13db47b9a95cffc2f8cbe51c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:21.451824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 859/1721] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:20 2022] DELETE /identity/v3/users/da0ffbdb13db47b9a95cffc2f8cbe51c => generated 0 bytes in 1065 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:21.456404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:21.457211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:21.457211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:21.546366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:21.548730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:21.691352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:21.694436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:21.840046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 863/1722] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:21.888318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59737e-82bd-4ec2-9b14-865328055362 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:21.891362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:21.893992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:21.908887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:21.909584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:21.909864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:21.910360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] PATH_INFO: `/v3/users/9f25b2bbb8d54599b1cf8a9f632d10aa` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:21.921026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f25b2bbb8d54599b1cf8a9f632d10aa)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:21.923331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:22.071825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f25b2bbb8d54599b1cf8a9f632d10aa'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:22.074587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f25b2bbb8d54599b1cf8a9f632d10aa'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:22.075559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f25b2bbb8d54599b1cf8a9f632d10aa'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:22.097563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f25b2bbb8d54599b1cf8a9f632d10aa'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:22.118025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8319f-99e0-450b-87fc-d5c1d1b6c1a3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:22.122965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:22.124669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.125332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.127258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.130720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:22.147316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-11510ae7-f49c-48d9-a8f1-185468375455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f25b2bbb8d54599b1cf8a9f632d10aa'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:22.168197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:22.174074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a95a6f-e80e-4952-926f-1097ef9adb4f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:22.175183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 860/1723] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:21 2022] DELETE /identity/v3/users/9f25b2bbb8d54599b1cf8a9f632d10aa => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:22.176655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547423-df50-4425-9a0b-650720c7c38a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.177226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547423-df50-4425-9a0b-650720c7c38a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.177559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20547423-df50-4425-9a0b-650720c7c38a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.179569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 861/1724] 10.222.0.26 () {54 vars in 912 bytes} [Mon Sep 12 23:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:53:22.181308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.184362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.184862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.224655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 864/1725] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:22.237283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.239302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.241135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.255778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:22.256834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:22.345171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:22.355900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:22.438598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:22.579564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 865/1726] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:53:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:22.607514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:22.619996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:22.685293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b50fd9-708b-4ab8-b04d-bba18890f11f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:22.717326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:22.729030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.731851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 862/1727] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:22.733894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.734179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.778319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:22.784226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:22.791109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:22.793315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:22.822299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 866/1728] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:22.887740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d697-9aa3-4cfd-a878-2794156d904d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:22.910019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:22.911381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:22.913595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:22.914126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:22.969968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:22.971708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:23.005529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 863/1729] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:23.096267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:23.098148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:23.251154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:23.376437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390207-9883-4d33-930b-fc55494fc0d1 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:23.402614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:23.403623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:23.404090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:23.404548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:23.429956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:23.540769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:23.547377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e6634-63fe-43db-8dfe-45385db0311e service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:23.622759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 867/1730] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:23.704818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:23.707370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:23.749040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dd38c-1ae5-460c-bc06-6653b6134cb1 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:23.786800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:23.788455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:23.789122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:23.796004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] PATH_INFO: `/v3/projects/15ce950d29b644bd84c483561209ce5d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:23.801824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15ce950d29b644bd84c483561209ce5d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:23.806849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:23.811599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:23.811902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:23.812160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:23.917473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] MFA Rules not processed for user `296f02c73fc548969bf60df4f8f372b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:23.921969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:24.047688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15ce950d29b644bd84c483561209ce5d', 'request_id': 'req-2a9e0678-0794-4290-83ef-01718b77c641'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.059301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15ce950d29b644bd84c483561209ce5d', 'request_id': 'req-2a9e0678-0794-4290-83ef-01718b77c641'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.086417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15ce950d29b644bd84c483561209ce5d', 'request_id': 'req-2a9e0678-0794-4290-83ef-01718b77c641'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.137255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] The token cache is being invalidate because project 15ce950d29b644bd84c483561209ce5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:24.144383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.150358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.153509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 864/1731] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:23 2022] DELETE /identity/v3/projects/15ce950d29b644bd84c483561209ce5d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:24.159445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:24.168257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:24.266141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 868/1732] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:24.291588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:24.294496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:24.476516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e0678-0794-4290-83ef-01718b77c641 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:24.489534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:24.490132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:24.490494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:24.490802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] PATH_INFO: `/v3/users/839a3d16c0e143b3b62846ae297ffdf5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:24.494808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=839a3d16c0e143b3b62846ae297ffdf5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:24.497214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:24.565011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94df808-c26d-42d3-9f0b-e2a7d35a7e1d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:24.636929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839a3d16c0e143b3b62846ae297ffdf5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.637667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839a3d16c0e143b3b62846ae297ffdf5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.638413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839a3d16c0e143b3b62846ae297ffdf5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.640829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:24.663480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839a3d16c0e143b3b62846ae297ffdf5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.718055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-aa380c20-128b-4ee0-ba4b-9ec05d08b754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839a3d16c0e143b3b62846ae297ffdf5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:24.739905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:24.777744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 865/1733] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:24 2022] DELETE /identity/v3/users/839a3d16c0e143b3b62846ae297ffdf5 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:24.812709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:24.816269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:24.819866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:24.820717 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:24.824806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:25.066816 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:25.075848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:25.083688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7812-f094-47dd-9d46-9f8ecaf98c1e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:25.084628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7812-f094-47dd-9d46-9f8ecaf98c1e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:25.085045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7812-f094-47dd-9d46-9f8ecaf98c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:25.130656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b62a7812-f094-47dd-9d46-9f8ecaf98c1e None None] MFA Rules not processed for user `76eb39c2010e43aaa0a01caaea3a38b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:25.134836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b62a7812-f094-47dd-9d46-9f8ecaf98c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:25.139670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 869/1734] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:25.455716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 866/1735] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:26.664691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:26.668361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:26.748726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b7aa-b1c3-4b71-acdd-c991cd7faf48 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:26.818048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:26.818580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:26.819290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:26.941274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] MFA Rules not processed for user `a8cbfc74644f4d4594a70b833ad7002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:26.946759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:27.124083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:27.130848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:27.142088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:27.143394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] PATH_INFO: `/v3/users/fe3f7fe0d41a4111b72fe0479fa2d55e` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:27.155130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3f7fe0d41a4111b72fe0479fa2d55e)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:27.172781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:27.374254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 867/1736] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 559 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:27.375315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3f7fe0d41a4111b72fe0479fa2d55e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:27.376284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3f7fe0d41a4111b72fe0479fa2d55e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:27.409629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:27.410414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3f7fe0d41a4111b72fe0479fa2d55e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:27.413149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:27.439913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3f7fe0d41a4111b72fe0479fa2d55e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:27.476478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cc57c000-55fa-40b8-9b32-04abdec57cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3f7fe0d41a4111b72fe0479fa2d55e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:27.496074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 870/1737] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:26 2022] DELETE /identity/v3/users/fe3f7fe0d41a4111b72fe0479fa2d55e => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:27.499647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86936b92-a992-4897-8b1a-ace480545913 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:27.499931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86936b92-a992-4897-8b1a-ace480545913 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:27.500205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86936b92-a992-4897-8b1a-ace480545913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:27.568683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-86936b92-a992-4897-8b1a-ace480545913 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:27.573736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-86936b92-a992-4897-8b1a-ace480545913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:27.599467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:27.777373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:27.818844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe583b-24a8-47e9-a0cf-38e46a7097fb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:27.874250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 871/1738] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:27.972225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86936b92-a992-4897-8b1a-ace480545913 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:27.974109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-86936b92-a992-4897-8b1a-ace480545913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:28.046217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:28.047146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:28.047601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:28.047997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:28.052775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:28.171627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86936b92-a992-4897-8b1a-ace480545913 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:28.189742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:28.192675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:28.194832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:28.195659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:28.253086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:28.258211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:28.288787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:28.290497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f524fd-83a9-4de3-9057-aef8aa692a85 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:28.320370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 872/1739] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:28.325780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 868/1740] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 918 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:28.367182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:28.368751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:28.404970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ed48-4cbe-4d22-bf2f-1d6da3b4c747 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:28.425149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:28.428224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:28.428712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:28.436639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] PATH_INFO: `/v3/projects/4d827105b9bd47a0bf020c56e7480d02` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:28.441979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d827105b9bd47a0bf020c56e7480d02)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:28.474094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:28.748391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d827105b9bd47a0bf020c56e7480d02', 'request_id': 'req-a0bc618b-3629-4acb-b1ae-f902504b10d4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:28.766836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d827105b9bd47a0bf020c56e7480d02', 'request_id': 'req-a0bc618b-3629-4acb-b1ae-f902504b10d4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:28.798914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d827105b9bd47a0bf020c56e7480d02', 'request_id': 'req-a0bc618b-3629-4acb-b1ae-f902504b10d4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:28.853647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] The token cache is being invalidate because project 4d827105b9bd47a0bf020c56e7480d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:28.854265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:28.858771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:28.865042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 873/1741] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:28 2022] DELETE /identity/v3/projects/4d827105b9bd47a0bf020c56e7480d02 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:28.870380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1313e-9c52-4597-b6ae-e9ec1bc2c1f3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:28.871256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1313e-9c52-4597-b6ae-e9ec1bc2c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:28.871256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1313e-9c52-4597-b6ae-e9ec1bc2c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:28.930219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-cbc1313e-9c52-4597-b6ae-e9ec1bc2c1f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:28.931946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-cbc1313e-9c52-4597-b6ae-e9ec1bc2c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:28.964312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:28.966926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:29.206686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:29.246781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 869/1742] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:29.249690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:29.250203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:29.250685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:29.353445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:29.355632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:29.358332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc618b-3629-4acb-b1ae-f902504b10d4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:29.378054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:29.379688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:29.381344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:29.382746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] PATH_INFO: `/v3/projects/0f1fcf9b05564a3487c7e85986d07567` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:29.386406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1fcf9b05564a3487c7e85986d07567)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:29.394494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:29.408645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 870/1743] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:29.414329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:29.469682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea67070-4d77-498e-a51f-1ff69eac2308 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:29.510773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:29.512555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:29.513314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:29.514031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:29.515624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:29.524242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:29.529865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:29.620393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 871/1744] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:29.622162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:29.635735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1fcf9b05564a3487c7e85986d07567', 'request_id': 'req-c0fe920a-d062-40e1-80b8-c84baf0e94ad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:29.677275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1fcf9b05564a3487c7e85986d07567', 'request_id': 'req-c0fe920a-d062-40e1-80b8-c84baf0e94ad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:29.695663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dcac8-4123-4c1a-9068-9bd8027f88a7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:29.713392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1fcf9b05564a3487c7e85986d07567', 'request_id': 'req-c0fe920a-d062-40e1-80b8-c84baf0e94ad'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:29.754532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] The token cache is being invalidate because project 0f1fcf9b05564a3487c7e85986d07567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:29.756229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:29.758931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:29.767427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 874/1745] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:28 2022] DELETE /identity/v3/projects/0f1fcf9b05564a3487c7e85986d07567 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:29.774251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:29.778192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:29.807390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:30.189042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe920a-d062-40e1-80b8-c84baf0e94ad admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:30.194166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:30.196349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:30.197896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:30.198710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:30.203531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:30.215915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:30.230706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:30.231675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:30.232064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:30.232532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] PATH_INFO: `/v3/users/ca962d04ed2b4208a0157ae4b04faf7c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:30.236125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca962d04ed2b4208a0157ae4b04faf7c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:30.239543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:30.410627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 872/1746] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:29 2022] POST /identity/v3/projects => generated 369 bytes in 789 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:30.416497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:30.421748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:30.544096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca962d04ed2b4208a0157ae4b04faf7c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:30.546774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca962d04ed2b4208a0157ae4b04faf7c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:30.634368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca962d04ed2b4208a0157ae4b04faf7c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:30.665014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca962d04ed2b4208a0157ae4b04faf7c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:30.695097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:30.764176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1f98dbce-3ac5-421d-b453-11ed5a7913c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca962d04ed2b4208a0157ae4b04faf7c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:30.797093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 875/1747] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:29 2022] DELETE /identity/v3/users/ca962d04ed2b4208a0157ae4b04faf7c => generated 0 bytes in 1025 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:30.849854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:30.850265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:30.850613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:30.902311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a45208-8d1e-4d33-a724-3ab08eed35e8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:31.078037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:31.079560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:31.303836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:31.305491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:31.310295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:31.310872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:31.315033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:31.319825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:31.437513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 876/1748] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:53:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 589 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 12 23:53:31.439545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 873/1749] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:30 2022] POST /identity/v3/users => generated 285 bytes in 1025 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:31.447868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:31.458202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:31.560397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b372e69-ca36-461f-9a02-59e7e4683d71 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:31.577951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:31.579087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:31.579634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:31.580107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:31.582052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:31.586403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:31.624741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 877/1750] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:31.627098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:31.668723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073679b-34e2-4227-8825-090029dea324 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:31.687267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:31.688431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:31.688980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:31.689436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:31.690596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:31.694986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:31.751060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 874/1751] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:31.754236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:31.813286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:31.816092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:31.829286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7af77-2cd3-43d7-b441-3dd6777782f0 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:31.855362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:31.857230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:31.858768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:31.860362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] PATH_INFO: `/v3/projects/53ae1785dac24e3b839023dcfa9a08bd/users/8df0b71918bd4fa8bd89610a5dfb6177/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:31.942136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:31.949323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ae1785dac24e3b839023dcfa9a08bd, user_id=8df0b71918bd4fa8bd89610a5dfb6177, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:31.969854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6756a-6dd3-4b6b-8a3b-c5f21924ae38 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:32.039426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 878/1752] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:31 2022] PUT /identity/v3/projects/53ae1785dac24e3b839023dcfa9a08bd/users/8df0b71918bd4fa8bd89610a5dfb6177/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:32.047190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:32.048093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:32.048579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:32.175877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] MFA Rules not processed for user `8df0b71918bd4fa8bd89610a5dfb6177`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:32.182143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:32.200350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ecb0d-c88e-4165-b530-aae37e820410 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:32.398623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 879/1753] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:32.402509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:32.448689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:32.540538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:32.541691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:32.542145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:32.542589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:32.725331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:32.727866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:32.751786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 875/1754] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:53:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 939 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:53:32.767740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918584b4-5912-4b6a-958a-047b0bd898f2 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:32.792420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:32.793593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:32.794067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:32.794550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:32.798003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:32.807836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:32.951159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 880/1755] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:32 2022] POST /identity/v3/projects => generated 369 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:32.957594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:33.106971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b87da8-3acd-4a12-8349-af636051a51a service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:33.583458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:33.663357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:33.664513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:33.674988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:33.675544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:33.681871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:33.698219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:33.845011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 876/1756] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:32 2022] POST /identity/v3/users => generated 291 bytes in 883 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:33.858361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:33.860669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870cff9-6ea3-4845-8469-5282adb39e70 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:33.877473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:33.878255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:33.878647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:33.879007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:33.879832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:33.883824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:33.924926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 881/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:33.931664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:33.951563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40202b8a-de01-4a72-bcbd-d553a9a37da5 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:33.979942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:33.980692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:33.981082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:33.981976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] PATH_INFO: `/v3/projects/c9e1e686012a4782a252a373a16a1d9b/users/e6ba0f638a26420389601cc184608bd6/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:34.029662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e1e686012a4782a252a373a16a1d9b, user_id=e6ba0f638a26420389601cc184608bd6, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:34.035616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:34.090855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 877/1758] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:33 2022] PUT /identity/v3/projects/c9e1e686012a4782a252a373a16a1d9b/users/e6ba0f638a26420389601cc184608bd6/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:34.101067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:34.147676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b62aa9-f757-41b4-b1bd-79c9fec11003 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:34.600571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:34.602413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:34.603172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:34.604872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:34.607304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:34.617662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bfddc-fd59-4c71-a1e5-59d021d98d20 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:34.671366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 882/1759] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:34.676805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:34.710841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fde32b-dd46-423a-956e-4c214af3ee84 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:34.735382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:34.737251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:34.738302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:34.741827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] PATH_INFO: `/v3/projects/c9e1e686012a4782a252a373a16a1d9b/users/e6ba0f638a26420389601cc184608bd6/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:34.766321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e1e686012a4782a252a373a16a1d9b, user_id=e6ba0f638a26420389601cc184608bd6, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:34.838275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:34.893851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 878/1760] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:34 2022] PUT /identity/v3/projects/c9e1e686012a4782a252a373a16a1d9b/users/e6ba0f638a26420389601cc184608bd6/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:34.912704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:34.914308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:34.915069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:35.036179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] MFA Rules not processed for user `e6ba0f638a26420389601cc184608bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:35.042906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:35.186505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:35.189534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:35.494120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 883/1761] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 583 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:35.513001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:35.515978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:35.542396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776eaae-d3cf-4b84-9762-ce8081c4dbd8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:35.552438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:35.569311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:35.576733 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:35.577399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:35.577863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] PATH_INFO: `/v3/projects/517176a7deb449fd8d17755dff6250d8` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:35.588490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517176a7deb449fd8d17755dff6250d8)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:35.592196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:35.846847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517176a7deb449fd8d17755dff6250d8', 'request_id': 'req-8901747f-534c-4ecc-89d3-9c9aa5d42292'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:35.877662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517176a7deb449fd8d17755dff6250d8', 'request_id': 'req-8901747f-534c-4ecc-89d3-9c9aa5d42292'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:35.918018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517176a7deb449fd8d17755dff6250d8', 'request_id': 'req-8901747f-534c-4ecc-89d3-9c9aa5d42292'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:35.965191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] The token cache is being invalidate because project 517176a7deb449fd8d17755dff6250d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:35.965512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:35.969701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:35.979659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 879/1762] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:35 2022] DELETE /identity/v3/projects/517176a7deb449fd8d17755dff6250d8 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:35.980344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:35.986272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:35.987766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3919e-8398-4f81-9ae3-88f35c7ebe73 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:36.149575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:36.284286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901747f-534c-4ecc-89d3-9c9aa5d42292 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:36.296786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:36.301149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:36.303410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:36.304700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:36.304993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] PATH_INFO: `/v3/users/05b1a38ac5e340d7aabd7d196c75abe3` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:36.308490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:36.308868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:36.309135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:36.311633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b1a38ac5e340d7aabd7d196c75abe3)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:36.317451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:36.317984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:36.484585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b1a38ac5e340d7aabd7d196c75abe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:36.485342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b1a38ac5e340d7aabd7d196c75abe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:36.485978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b1a38ac5e340d7aabd7d196c75abe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:36.547925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b1a38ac5e340d7aabd7d196c75abe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:36.588356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:36.589966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:36.622987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b090e1df-4995-4702-803c-e66f50af233f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b1a38ac5e340d7aabd7d196c75abe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:36.645177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 884/1763] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 1132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:36.696843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 880/1764] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:35 2022] DELETE /identity/v3/users/05b1a38ac5e340d7aabd7d196c75abe3 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:37.546410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:37.549324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:37.597286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90689ad6-97f4-484e-b36a-c2768110cde4 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:37.793634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:37.794611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:37.795028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:37.795439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] PATH_INFO: `/v3/projects/26f349c524f24c848a43920142474e53` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:37.805206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f349c524f24c848a43920142474e53)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:37.807619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:37.894512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:38.028586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f349c524f24c848a43920142474e53', 'request_id': 'req-794d1d23-435b-4afe-8888-a9ad694003b1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:38.060762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f349c524f24c848a43920142474e53', 'request_id': 'req-794d1d23-435b-4afe-8888-a9ad694003b1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:38.083326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f349c524f24c848a43920142474e53', 'request_id': 'req-794d1d23-435b-4afe-8888-a9ad694003b1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:38.124922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] The token cache is being invalidate because project 26f349c524f24c848a43920142474e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:38.125839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:38.128211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-794d1d23-435b-4afe-8888-a9ad694003b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:38.141817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 885/1765] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:37 2022] DELETE /identity/v3/projects/26f349c524f24c848a43920142474e53 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:38.148706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:38.150220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:38.150968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:38.256621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:38.265585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:38.589588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 881/1766] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:38.591136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:38.591798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:38.592358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:38.697355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:38.713882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:38.758117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 886/1767] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:38.763556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:38.789797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd6e40-f332-4bbb-a032-9f8d86242fed None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:38.814250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:38.820360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:38.823012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:38.823783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:38.829209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:38.831766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:38.841655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:38.950066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:38.955899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 882/1768] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:38.964236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ceeee-4c0c-4029-bd74-652da434e617 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:38.981931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:38.982866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:38.983337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:38.983767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:38.984629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:38.987014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:39.058073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 887/1769] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:38 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:39.063558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:39.077324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade859-128f-4fc0-baf2-919efd3128b1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:39.249188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:39.289176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:39.292887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:39.293473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:39.294021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:39.295113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:39.299597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:39.347939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 883/1770] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:39 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:39.351934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:39.373668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0356683-3ea1-4946-a21d-0b905bdfe2aa admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:39.389015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:39.389349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:39.390071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:39.391727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:39.392851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:39.398762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:39.429568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 888/1771] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:39.434945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:39.452989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be201f-71f6-45b9-9183-469110d3eaae admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:39.482720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:39.485945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:39.487655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:39.487951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:39.491505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:39.501866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154832ac-5278-423b-9238-0787be1656e5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:39.555559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 884/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:39.561942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:39.579763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ec1c7-2aa8-46d8-9594-8bd38e98d2bd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:39.593718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:39.596697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:39.597416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:39.599353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] PATH_INFO: `/v3/projects/b248d3a73c8b4361b2375aafee2508d6/users/b36a2693c1134709888d14b284f45093/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:39.678608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b248d3a73c8b4361b2375aafee2508d6, user_id=b36a2693c1134709888d14b284f45093, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:39.681878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:39.717730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 889/1773] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:39 2022] PUT /identity/v3/projects/b248d3a73c8b4361b2375aafee2508d6/users/b36a2693c1134709888d14b284f45093/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:39.749809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82f2-157d-434b-9086-cb4b016cc4d0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:39.750391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82f2-157d-434b-9086-cb4b016cc4d0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:39.750837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82f2-157d-434b-9086-cb4b016cc4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:39.819358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4f5f82f2-157d-434b-9086-cb4b016cc4d0 None None] MFA Rules not processed for user `b36a2693c1134709888d14b284f45093`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:39.820969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4f5f82f2-157d-434b-9086-cb4b016cc4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:40.298997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 885/1774] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 550 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:40.314030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:40.316165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:40.472980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34672-da88-4fb8-b9cd-8ea8ddbb73f3 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:40.483380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:40.484318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:40.484830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:40.485204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:40.502783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:40.504527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:40.533097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 890/1775] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:40.757750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3832a36-41e4-4ba3-8a7c-6dd9360005e8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:40.758241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3832a36-41e4-4ba3-8a7c-6dd9360005e8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:40.758730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3832a36-41e4-4ba3-8a7c-6dd9360005e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:40.814816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c3832a36-41e4-4ba3-8a7c-6dd9360005e8 None None] MFA Rules not processed for user `a8cbfc74644f4d4594a70b833ad7002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:40.820015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c3832a36-41e4-4ba3-8a7c-6dd9360005e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:40.822201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:40.826884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:41.060412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:41.194232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2538b-0b1a-45d5-b28f-56b97183859a service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:41.202453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 886/1776] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:41.225778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:41.255130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:41.255130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:41.255130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] PATH_INFO: `/v3/projects/c4762704abee4fd095ece719ee0eab6c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:41.304175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4762704abee4fd095ece719ee0eab6c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:41.317871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:41.579871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4762704abee4fd095ece719ee0eab6c', 'request_id': 'req-1258c400-cde9-453e-9bfd-52525d64b129'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:41.633866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4762704abee4fd095ece719ee0eab6c', 'request_id': 'req-1258c400-cde9-453e-9bfd-52525d64b129'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:41.651316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4762704abee4fd095ece719ee0eab6c', 'request_id': 'req-1258c400-cde9-453e-9bfd-52525d64b129'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:41.755553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] The token cache is being invalidate because project c4762704abee4fd095ece719ee0eab6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:41.756257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:41.758241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-1258c400-cde9-453e-9bfd-52525d64b129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:41.768561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:41.769278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:41.769533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:41.770469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 891/1777] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:40 2022] DELETE /identity/v3/projects/c4762704abee4fd095ece719ee0eab6c => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:41.858709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:41.860776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:41.994187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:42.028216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 887/1778] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:42.038334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.041430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.042110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.074816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:42.076600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.106908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df242eb5-1b89-42c6-97dd-2443482a8612 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:42.109723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:42.111843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.134140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:42.135310 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.135829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.136332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] PATH_INFO: `/v3/users/8df0b71918bd4fa8bd89610a5dfb6177` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.142962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df0b71918bd4fa8bd89610a5dfb6177)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:42.145082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:42.150637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 892/1779] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:42.155008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:42.181217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9ba15-288c-4a7f-a8ec-2a851ea687e8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:42.206534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:42.210130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.210432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.210686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.212707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:42.213279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:42.215191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f1338-e0c9-441d-bee7-ba8cb84f017b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:42.275171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 893/1780] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:42.278375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1b93-8866-47ca-9a10-b3650aa8f1d9 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.280987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1b93-8866-47ca-9a10-b3650aa8f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.282165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb1b93-8866-47ca-9a10-b3650aa8f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.303409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df0b71918bd4fa8bd89610a5dfb6177'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:42.305320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df0b71918bd4fa8bd89610a5dfb6177'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:42.306890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df0b71918bd4fa8bd89610a5dfb6177'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:42.334716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9dbb1b93-8866-47ca-9a10-b3650aa8f1d9 None None] MFA Rules not processed for user `296f02c73fc548969bf60df4f8f372b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:42.339739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9dbb1b93-8866-47ca-9a10-b3650aa8f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.352293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df0b71918bd4fa8bd89610a5dfb6177'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:42.408178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df0b71918bd4fa8bd89610a5dfb6177'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:42.462812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 888/1781] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:42 2022] DELETE /identity/v3/users/8df0b71918bd4fa8bd89610a5dfb6177 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:42.465017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:42.467504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.598727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 894/1782] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:42.598727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.598727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.598727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.624311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:42.625890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.710293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea4192-44cb-4bcf-8e23-0a889994fe95 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:42.756756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:42.759711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:42.763158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:42.765076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:42.765219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:42.767052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:42.781191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:42.812428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 895/1783] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:42.831582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:42.833470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:42.888545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 889/1784] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:42 2022] POST /identity/v3/projects => generated 393 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:42.896898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:43.079065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43358917-fa78-4377-a1ab-b6e187dd6720 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:43.164181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:43.257723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d280890-b29c-4d89-81fc-d8013cf0a3d4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:43.276213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:43.277595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.278309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.281723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.283322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:43.289331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d15ff-bbb2-4d27-913e-9169487f1382 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:43.367177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 890/1785] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:42 2022] POST /identity/v3/users => generated 297 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:43.369804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.370489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.371353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.434593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:43.443656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.444289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.444750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.449475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:43.451632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:43.466564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 891/1786] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Sep 12 23:53:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:43.467946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:43.490115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349ce74-a127-42a4-a5d6-60d6c814b9eb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:43.497161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:43.499180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:43.502413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:43.503632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.504201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.504966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.506219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:43.509469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:43.531775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 892/1787] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:43.536982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:43.549272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 896/1788] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:43.558232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d1a62-c2a4-4d83-94f3-a8d49108e016 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:43.569766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:43.572612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.573310 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.573892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.575246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:43.579311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e949c6d-c8a9-4e27-860a-c20f4a06d984 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:43.611434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 893/1789] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:43.616206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:43.631400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fee34-3d45-4c97-b69e-dc97b67a4a2a service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:43.637413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:43.638765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.639061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.639346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] PATH_INFO: `/v3/projects/129f5ff9fab74eaebfc211cf7c81ab0e/users/63ebf576d0eb4972b407044bdeb13d94/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.672429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=129f5ff9fab74eaebfc211cf7c81ab0e, user_id=63ebf576d0eb4972b407044bdeb13d94, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:43.676657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:43.722260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 897/1790] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:43 2022] PUT /identity/v3/projects/129f5ff9fab74eaebfc211cf7c81ab0e/users/63ebf576d0eb4972b407044bdeb13d94/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:43.734689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:43.735175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:43.735640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:43.866771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] MFA Rules not processed for user `63ebf576d0eb4972b407044bdeb13d94`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:43.872329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:44.198122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 894/1791] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:44.205579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:44.369231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:44.370866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:44.519778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac2a0e-70b7-4415-a965-e62564ee8c0d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:44.541267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:44.541539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:44.541883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:44.542128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:44.542808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:44.544858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:44.588862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 898/1792] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:44 2022] POST /identity/v3/projects => generated 393 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:44.594507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:44.620233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543df50-ec29-4270-bdb2-7b9e8b5c0ec5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:44.652120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:44.752210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:44.775846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:44.777108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:44.777626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:44.778105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] PATH_INFO: `/v3/projects/53ae1785dac24e3b839023dcfa9a08bd` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:44.779709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ae1785dac24e3b839023dcfa9a08bd)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:44.782418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:44.815056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad41fe3-cf0e-4ebc-87b2-127d7fe89bc4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:44.836119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:44.836788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:44.837069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:44.837441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:44.842837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:44.848787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:44.935572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 899/1793] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:44 2022] POST /identity/v3/users => generated 303 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:44.941301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:44.966962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2029420-2f63-4965-8a30-86220b1122b3 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:44.970020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ae1785dac24e3b839023dcfa9a08bd', 'request_id': 'req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:44.983811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ae1785dac24e3b839023dcfa9a08bd', 'request_id': 'req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.016120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ae1785dac24e3b839023dcfa9a08bd', 'request_id': 'req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.076721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:45.077845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.078110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.078706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:45.079623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:45.081704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:45.085642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] The token cache is being invalidate because project 53ae1785dac24e3b839023dcfa9a08bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:45.086153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.090963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.097621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 900/1794] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:45.107949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 895/1795] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:44 2022] DELETE /identity/v3/projects/53ae1785dac24e3b839023dcfa9a08bd => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:45.108153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:45.109886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:45.110246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:45.115150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:45.152347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f759bb3-86c2-42ed-99e9-8490affd9405 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:45.167442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:45.168157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.168439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.172906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] PATH_INFO: `/v3/projects/770acc5d03d84611b4e49a4b18819c29/users/215419b9dd9947c98f7f9b7129a31e98/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:45.189334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e0ff7-5cb4-4b4a-8a5b-a3ce0e58e91f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:45.202672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:45.213843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.214734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.215372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] PATH_INFO: `/v3/users/e6ba0f638a26420389601cc184608bd6` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:45.218409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ba0f638a26420389601cc184608bd6)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:45.220066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=770acc5d03d84611b4e49a4b18819c29, user_id=215419b9dd9947c98f7f9b7129a31e98, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:45.224184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:45.225485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:45.264656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 901/1796] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:45 2022] PUT /identity/v3/projects/770acc5d03d84611b4e49a4b18819c29/users/215419b9dd9947c98f7f9b7129a31e98/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:45.270654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:45.422437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ba0f638a26420389601cc184608bd6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.434680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ba0f638a26420389601cc184608bd6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.436062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ba0f638a26420389601cc184608bd6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.451743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ba0f638a26420389601cc184608bd6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.476285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1daf0b-1b2e-4338-b1e2-9ddd5fce76bb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:45.482689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ba0f638a26420389601cc184608bd6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:45.497907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:45.498968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.499459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.499909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:45.501027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:45.503832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc46d1a-cf3b-4816-adb0-de09509f45fa admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:45.523409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 896/1797] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:45 2022] DELETE /identity/v3/users/e6ba0f638a26420389601cc184608bd6 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:45.573634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 902/1798] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:53:45.582512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:45.586921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:45.719894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:45.820430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a87c4-b0b9-4044-9dc6-412e22cca96f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:45.840645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:45.845998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.845998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.845998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] PATH_INFO: `/v3/projects/770acc5d03d84611b4e49a4b18819c29/users/215419b9dd9947c98f7f9b7129a31e98/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:45.858119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=770acc5d03d84611b4e49a4b18819c29, user_id=215419b9dd9947c98f7f9b7129a31e98, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:45.870761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:45.935443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 897/1799] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:45 2022] PUT /identity/v3/projects/770acc5d03d84611b4e49a4b18819c29/users/215419b9dd9947c98f7f9b7129a31e98/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:45.946631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:45.947029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:45.947507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:46.074887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] MFA Rules not processed for user `215419b9dd9947c98f7f9b7129a31e98`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:46.086542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.107626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:46.117565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:46.120242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.400353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 903/1800] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:46.400353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:46.403286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.470029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2c9f-bf67-458b-8194-7ba38fcbb722 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:46.500405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:46.502528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:46.502979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:46.515086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:46.523632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.547549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:46.549989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:46.570768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 898/1801] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:46.571598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:46.573745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.611732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab552-8d41-4866-8989-de99faf36481 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:46.618581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:46.621425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:46.621836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:46.622109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:46.648529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3dbbf-d815-4794-81f0-291c8855766f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:46.657992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:46.660140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:46.668330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:46.668991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:46.669272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:46.669545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:46.695983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 899/1802] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:46.697924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:46.700052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:46.700910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.704207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:46.722533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 904/1803] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:46.729539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:46.732828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:46.893342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf84f06-87b6-4664-94e8-ebb27daad4ee service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:46.900059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:46.900059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:46.900059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:46.900059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:46.916232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:46.918136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:46.934842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 900/1804] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:47.090857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07f6f7-b2f4-4209-a5fe-2736b40ea471 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:47.112777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:47.114324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:47.115054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:47.115725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] PATH_INFO: `/v3/projects/c9e1e686012a4782a252a373a16a1d9b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:47.119146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e1e686012a4782a252a373a16a1d9b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:47.127524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:47.259929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:47.295862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c627ee-07f2-4a93-a06a-2b8586c0f074 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:47.318753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:47.321548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:47.322518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:47.323821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:47.336061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:47.362625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:47.410275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e1e686012a4782a252a373a16a1d9b', 'request_id': 'req-007c4b8c-78b3-4403-81a2-6cdddc023c1d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:47.440905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e1e686012a4782a252a373a16a1d9b', 'request_id': 'req-007c4b8c-78b3-4403-81a2-6cdddc023c1d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:47.474927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e1e686012a4782a252a373a16a1d9b', 'request_id': 'req-007c4b8c-78b3-4403-81a2-6cdddc023c1d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:47.523050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] The token cache is being invalidate because project c9e1e686012a4782a252a373a16a1d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:47.523735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:47.528018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-007c4b8c-78b3-4403-81a2-6cdddc023c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:47.533295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 905/1805] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:46 2022] DELETE /identity/v3/projects/c9e1e686012a4782a252a373a16a1d9b => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:47.541216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:47.541691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:47.542170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:47.670029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:47.673601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:47.694185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:47.697668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d98f3f-83b7-4ec2-9189-d4a710ace0c5 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:47.737762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 901/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:48.002962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 906/1807] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:48.020070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:48.020595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:48.021105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:48.134552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:48.136744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:48.181874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 902/1808] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:48.191654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:48.230932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe56b53-ca33-4a0c-9e4d-fc1547405c7d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:48.241326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:48.243053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:48.243983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:48.244689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:48.246630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:48.247377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:48.251362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:48.319819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 907/1809] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:48.326616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:48.350101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb470-bfd5-4092-8cd2-6b146182ef93 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:48.406685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:48.411287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:48.412466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:48.413586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:48.421455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:48.427338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:48.478618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:48.523002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 903/1810] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:48 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:48.529496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:48.573204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf7020-3efd-49f1-9ec5-73dc0c4410dc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:48.783570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:48.784897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:48.785385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:48.786504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:48.789025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:48.794164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:48.860074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 908/1811] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:48 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:48.867166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:48.899166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604523-ee5a-46fc-b448-8e3f259c8d55 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:48.924571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:48.926162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:48.926162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:48.927417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:48.929105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:48.959540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:49.009538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 904/1812] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:49.014214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:49.044816 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb482-c4a6-42be-9c5d-3db817f3fd50 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:49.059662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:49.059662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:49.059662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:49.059662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:49.060860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:49.063411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fe4c4-3e96-4a1d-88f9-24ba766fce99 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:49.104552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 909/1813] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:49.111278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:49.145149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744803ca-c41c-4737-885b-a946b7671fa3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:49.166374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:49.168926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:49.169762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:49.170287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] PATH_INFO: `/v3/projects/59eba439a2db4945bac6a5a16f44b758/users/f086e68fe98a4fdf8d35fcef506f5d42/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:49.240495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59eba439a2db4945bac6a5a16f44b758, user_id=f086e68fe98a4fdf8d35fcef506f5d42, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:49.272001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5cdf8-5dc7-414d-a6a1-f88115fe98b0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:49.361068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 905/1814] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:49 2022] PUT /identity/v3/projects/59eba439a2db4945bac6a5a16f44b758/users/f086e68fe98a4fdf8d35fcef506f5d42/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:49.376230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27f513-2b8f-45b4-8370-27f0435e59b2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:49.376953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27f513-2b8f-45b4-8370-27f0435e59b2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:49.378331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27f513-2b8f-45b4-8370-27f0435e59b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:49.526242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7c27f513-2b8f-45b4-8370-27f0435e59b2 None None] MFA Rules not processed for user `f086e68fe98a4fdf8d35fcef506f5d42`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:49.537564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7c27f513-2b8f-45b4-8370-27f0435e59b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:49.684587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:49.867935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c880a9-f09e-40f7-a670-4f3f0295d26d None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:49.868527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c880a9-f09e-40f7-a670-4f3f0295d26d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:49.869095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c880a9-f09e-40f7-a670-4f3f0295d26d None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:49.871962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 906/1815] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:53:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:53:49.887067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 910/1816] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:49.892769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c880a9-f09e-40f7-a670-4f3f0295d26d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:50.013460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:50.013845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:50.014581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:50.077037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:50.083525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:50.120678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c880a9-f09e-40f7-a670-4f3f0295d26d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:50.136447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:50.137151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:50.138629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:50.138995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:50.144836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:50.150570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:50.255323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 907/1817] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:49 2022] POST /identity/v3/projects => generated 387 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:50.260466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:50.344519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 911/1818] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:53:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:50.602915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531334-a7aa-4a29-b907-3c06889d210d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:50.635807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:50.637581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:50.645332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:50.646012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:50.650869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:50.666776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:50.794368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:50.797433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:50.854680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 908/1819] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:50 2022] POST /identity/v3/users => generated 300 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:53:50.866129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:50.948631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628ce2a-c1e0-4d96-95ad-e5bbb8d2da47 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:50.972984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c07fa-5cc8-4eec-9174-f990bd6c431f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:51.030814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:51.041295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.041847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.062608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.071115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:51.076185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:51.131814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 909/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:51.138971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:51.224597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e70f-c1ac-439a-ae93-cfe0f92d632b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:51.251140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:51.254369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.255061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.255577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] PATH_INFO: `/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d/users/49671294fcf64c9d8d4a59d4bc8eb785/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.336573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:51.346727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5278dc1b53fc49d28a74d7dc857fa88d, user_id=49671294fcf64c9d8d4a59d4bc8eb785, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:51.356419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:51.364869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:51.365577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.368184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.368582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.390324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 910/1821] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:51 2022] PUT /identity/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d/users/49671294fcf64c9d8d4a59d4bc8eb785/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:51.396961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:51.408999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:51.412433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:51.470284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 912/1822] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:51.558623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8ff42-41ec-4257-a726-6aafc05f3e7a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:51.566776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:51.567632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.568088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.568405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.569133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:51.570938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9665848-1e48-492c-b9eb-c3bb5080ee19 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:51.589041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 911/1823] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:51.593332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:51.611634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61b103-235c-44c2-8880-26b4e5e2fec3 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:51.618427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:51.619418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.619761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.620179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] PATH_INFO: `/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d/users/49671294fcf64c9d8d4a59d4bc8eb785/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.623182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5278dc1b53fc49d28a74d7dc857fa88d, user_id=49671294fcf64c9d8d4a59d4bc8eb785, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:51.627005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:51.677523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 913/1824] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:51 2022] PUT /identity/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d/users/49671294fcf64c9d8d4a59d4bc8eb785/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:51.690255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:51.690960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:51.690960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:51.734379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] MFA Rules not processed for user `49671294fcf64c9d8d4a59d4bc8eb785`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:51.735700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:51.873983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 912/1825] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:51.885893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:51.889031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:51.974861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:52.038350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ee5b1-58db-4ce2-8fe5-a09302f7b298 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:52.047260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:52.048433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.048917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.049400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.079076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:52.080933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25fc9b-ebf2-4094-b383-b0bf9c92baf2 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:52.093394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 914/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:52.128541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:52.240442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac62c-62ce-4cba-a456-52ca9b1595d6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.240706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac62c-62ce-4cba-a456-52ca9b1595d6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.240878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17764e8c-13a1-4892-a9db-a4b20f8b8613 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:52.241041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac62c-62ce-4cba-a456-52ca9b1595d6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.242256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 915/1827] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:53:52.249186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:52.250280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.250710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.251530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] PATH_INFO: `/v3/users/f086e68fe98a4fdf8d35fcef506f5d42` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.254128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f086e68fe98a4fdf8d35fcef506f5d42)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:52.258149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:52.275434 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.275823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.276054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.289463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:52.310305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:52.312859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:52.364996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f086e68fe98a4fdf8d35fcef506f5d42'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:52.366846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f086e68fe98a4fdf8d35fcef506f5d42'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:52.368338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f086e68fe98a4fdf8d35fcef506f5d42'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:52.390824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f086e68fe98a4fdf8d35fcef506f5d42'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:52.423570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b2c8c937-06b5-40d0-a0c7-bebdeec0d0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f086e68fe98a4fdf8d35fcef506f5d42'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:52.451281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 913/1828] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:52 2022] DELETE /identity/v3/users/f086e68fe98a4fdf8d35fcef506f5d42 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:52.455504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.455788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.456063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.478718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:52.480066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:52.486473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 916/1829] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:53:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:52.506775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:52.508445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:52.525763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a08a9f-4ae8-470e-902b-cc093b286939 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:52.555054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 914/1830] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:53:52.568568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:52.570814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:52.619520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187931fb-a153-411f-806e-fa652725af6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:52.620395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187931fb-a153-411f-806e-fa652725af6b service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.620832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187931fb-a153-411f-806e-fa652725af6b service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.621179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187931fb-a153-411f-806e-fa652725af6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.636411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187931fb-a153-411f-806e-fa652725af6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:52.639059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187931fb-a153-411f-806e-fa652725af6b service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:52.656980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 917/1831] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:52.657164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f285-0ef7-4669-b0e9-13d31cc94095 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:52.662966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:52.664130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:52.664575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:52.665025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:52.681635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:52.684258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:52.702042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 915/1832] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:53.411270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187931fb-a153-411f-806e-fa652725af6b service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:53.413064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-187931fb-a153-411f-806e-fa652725af6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:53.436325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187931fb-a153-411f-806e-fa652725af6b service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:53.465499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:53.467081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:53.467983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:53.469213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] PATH_INFO: `/v3/projects/59eba439a2db4945bac6a5a16f44b758` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:53.472361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59eba439a2db4945bac6a5a16f44b758)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:53.476836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:53.544598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:53.547153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59eba439a2db4945bac6a5a16f44b758', 'request_id': 'req-b366e70a-9057-40af-9ccd-672b6db431f2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.558202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59eba439a2db4945bac6a5a16f44b758', 'request_id': 'req-b366e70a-9057-40af-9ccd-672b6db431f2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.567159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59eba439a2db4945bac6a5a16f44b758', 'request_id': 'req-b366e70a-9057-40af-9ccd-672b6db431f2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.587119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] The token cache is being invalidate because project 59eba439a2db4945bac6a5a16f44b758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:53.587623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.588713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.591553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 918/1833] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:53 2022] DELETE /identity/v3/projects/59eba439a2db4945bac6a5a16f44b758 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:53.595581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:53.607116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406395a-d313-423c-b7c1-a5d31ce7e8ed service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:53.656712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:53.704191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:53.705347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:53.705840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:53.706320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] PATH_INFO: `/v3/users/49671294fcf64c9d8d4a59d4bc8eb785` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:53.708596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49671294fcf64c9d8d4a59d4bc8eb785)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:53.712674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:53.773710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49671294fcf64c9d8d4a59d4bc8eb785'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.774447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49671294fcf64c9d8d4a59d4bc8eb785'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.775175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49671294fcf64c9d8d4a59d4bc8eb785'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.784508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49671294fcf64c9d8d4a59d4bc8eb785'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.799027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49671294fcf64c9d8d4a59d4bc8eb785'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:53.816525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 916/1834] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:53 2022] DELETE /identity/v3/users/49671294fcf64c9d8d4a59d4bc8eb785 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:54.609077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:54.615608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:54.685077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366e70a-9057-40af-9ccd-672b6db431f2 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:54.928868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:54.928868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:54.928868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:54.928868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] PATH_INFO: `/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:54.930003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5278dc1b53fc49d28a74d7dc857fa88d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:54.934879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.041026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5278dc1b53fc49d28a74d7dc857fa88d', 'request_id': 'req-8b11e768-add3-4120-acdd-aabaec06e608'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.049987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5278dc1b53fc49d28a74d7dc857fa88d', 'request_id': 'req-8b11e768-add3-4120-acdd-aabaec06e608'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.059510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.064354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:55.066392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5278dc1b53fc49d28a74d7dc857fa88d', 'request_id': 'req-8b11e768-add3-4120-acdd-aabaec06e608'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.087654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] The token cache is being invalidate because project 5278dc1b53fc49d28a74d7dc857fa88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:55.088263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c4d76-179d-483b-adb8-dbff276c4514 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.088423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.089491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8b11e768-add3-4120-acdd-aabaec06e608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.093026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 919/1835] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:54 2022] DELETE /identity/v3/projects/5278dc1b53fc49d28a74d7dc857fa88d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:55.097162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd852c-ed52-488a-be41-47a8747db949 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.097588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd852c-ed52-488a-be41-47a8747db949 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.097961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd852c-ed52-488a-be41-47a8747db949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.129438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7ecd852c-ed52-488a-be41-47a8747db949 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:55.131382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7ecd852c-ed52-488a-be41-47a8747db949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:55.193304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.194350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.194813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.195288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] PATH_INFO: `/v3/users/63ebf576d0eb4972b407044bdeb13d94` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.201978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ebf576d0eb4972b407044bdeb13d94)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.204191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.251503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 920/1836] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:55.257214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.258508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.258972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.295117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ebf576d0eb4972b407044bdeb13d94'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.307493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ebf576d0eb4972b407044bdeb13d94'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.307493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ebf576d0eb4972b407044bdeb13d94'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.307684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:55.307778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ebf576d0eb4972b407044bdeb13d94'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.309991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:55.315319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:55.318232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:55.329481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3565ab8e-e3de-4b2e-a010-8108da068465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ebf576d0eb4972b407044bdeb13d94'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:55.333144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 921/1837] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:55.338394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.351491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 917/1838] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:55 2022] DELETE /identity/v3/users/63ebf576d0eb4972b407044bdeb13d94 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:55.353192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43335da-d8ab-48a6-b8a6-0ad5a1a42ec8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.356469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.356826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.357153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.387033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:55.388574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:55.440618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.441731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.442561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.443140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.444433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:53:55.444997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.448258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.460863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 918/1839] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:55.474347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 922/1840] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:53:55.480448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.482109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:55.492273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.533048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b890b-2fb1-486e-92a1-310ea47380fc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.570087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.571370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.573954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.574912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.576746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.585198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.640178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd7a9f-3fd1-44bc-96c3-8994382fa548 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.656426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.656426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.656426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.656426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.678590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.680276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.690772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 923/1841] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:55 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:55.693184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.724550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 919/1842] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:55.823246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62ce4-e023-4bed-afb8-01b1dcd6f354 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.829314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.829912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.830199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.830470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.831175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.833325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.899787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 924/1843] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:55 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:55.906187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:55.922636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82359e6b-9e02-4d16-84df-ccf825cc2d06 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:55.932478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:55.933051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:55.933330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:55.933598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:55.934288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:55.945871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:55.982020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 920/1844] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:55.988128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.000413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2474-338e-45e7-8ec0-b0d122500a69 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.006413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.007153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.007589 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.007931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.009002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.011748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc70eea-f484-40c6-a827-57db90d661c0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.024455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 925/1845] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:56.030573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.056275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f9f50-a942-4962-9fb9-24e7601fde7c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.063888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.065033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.065833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.066451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] PATH_INFO: `/v3/projects/20b8940301a941f683e94d70bd31ae60/users/715a07cb193d4f80a6d860144d037d50/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.097142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b8940301a941f683e94d70bd31ae60, user_id=715a07cb193d4f80a6d860144d037d50, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.101544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.123567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 921/1846] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:56 2022] PUT /identity/v3/projects/20b8940301a941f683e94d70bd31ae60/users/715a07cb193d4f80a6d860144d037d50/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:56.137329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30356b89-e283-4973-a946-52791e155524 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.137329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30356b89-e283-4973-a946-52791e155524 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.137329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30356b89-e283-4973-a946-52791e155524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.218711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-30356b89-e283-4973-a946-52791e155524 None None] MFA Rules not processed for user `715a07cb193d4f80a6d860144d037d50`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:56.221905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-30356b89-e283-4973-a946-52791e155524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:56.335016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:56.395294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 926/1847] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:56.407830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.492437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:56.531472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895c56-5023-4f7f-abde-bf5797f19e94 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.539773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.540941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.541577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.542274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.543360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.545920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.627584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 922/1848] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:53:56 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:56.632509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30356b89-e283-4973-a946-52791e155524 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.674943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30356b89-e283-4973-a946-52791e155524 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.784115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.785161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.785161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.785681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.786138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.788426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.827776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 927/1849] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:53:56 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:53:56.830824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.852213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d268a3-5e63-4cbe-9525-440a9f822abe admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.860595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.861427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.861703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.862116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.862792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.864679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.879271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 923/1850] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:56.883136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.896333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bc687-0842-4208-96a2-28a507896ab9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:56.902196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:56.903154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:56.903585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:56.904007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] PATH_INFO: `/v3/projects/e9cf28f01e224b218cec4f3b6581489f/users/2812410eefe3487e8eb5edb47b118e18/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:56.928031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cf28f01e224b218cec4f3b6581489f, user_id=2812410eefe3487e8eb5edb47b118e18, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:56.930902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:56.953799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 928/1851] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:56 2022] PUT /identity/v3/projects/e9cf28f01e224b218cec4f3b6581489f/users/2812410eefe3487e8eb5edb47b118e18/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:56.958593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:56.972574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363d44e-1dc5-4008-bde5-6c15bf49a4ac admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:57.064848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:57.066061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:57.066612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:57.069266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:57.071042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:57.073357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb214-7cf8-4108-9dd0-ae2c7cfac101 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:57.085184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 924/1852] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:53:57.090419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:57.106574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ad0a-fdb1-476e-8687-5bcd3c2dc7cc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:57.112376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:57.113291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:57.113612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:57.113940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] PATH_INFO: `/v3/projects/e9cf28f01e224b218cec4f3b6581489f/users/2812410eefe3487e8eb5edb47b118e18/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:57.116389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cf28f01e224b218cec4f3b6581489f, user_id=2812410eefe3487e8eb5edb47b118e18, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:57.120053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:57.133878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 929/1853] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:53:57 2022] PUT /identity/v3/projects/e9cf28f01e224b218cec4f3b6581489f/users/2812410eefe3487e8eb5edb47b118e18/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:57.145051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:57.145505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:57.145841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:57.184158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] MFA Rules not processed for user `2812410eefe3487e8eb5edb47b118e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:57.185398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:57.302433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 925/1854] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:57.320795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:57.330407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:57.513425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afcf21-75fe-4da8-9499-8ffa6ccfef11 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:57.530705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:57.533143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:57.533558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:57.535810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:57.559459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:57.563684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fc252-eec0-4545-8e2d-d673ee97f0d1 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:57.601913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 930/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:53:57.979192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:58.079982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:58.161829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fd157-d56b-445a-93e6-d56501eb86da None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:58.173735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:58.174102 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:58.175844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:58.176892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] PATH_INFO: `/v3/users/715a07cb193d4f80a6d860144d037d50` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:58.180711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=715a07cb193d4f80a6d860144d037d50)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:58.186570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:58.288049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715a07cb193d4f80a6d860144d037d50'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:58.289139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715a07cb193d4f80a6d860144d037d50'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:58.291761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715a07cb193d4f80a6d860144d037d50'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:58.306076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715a07cb193d4f80a6d860144d037d50'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:58.366004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715a07cb193d4f80a6d860144d037d50'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:58.422567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 926/1856] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:57 2022] DELETE /identity/v3/users/715a07cb193d4f80a6d860144d037d50 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:53:58.424542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07577483-56b1-4c08-948c-9532771b7edb None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:58.424991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07577483-56b1-4c08-948c-9532771b7edb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:58.425090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07577483-56b1-4c08-948c-9532771b7edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:58.488856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-07577483-56b1-4c08-948c-9532771b7edb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:53:58.494129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-07577483-56b1-4c08-948c-9532771b7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:58.580508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:58.646459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 931/1857] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:53:58.741175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:58.742684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:58.823969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e714-df3c-4f0a-ac5d-6a91d5f12a3b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:58.830284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:58.830969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:58.831501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:58.831647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:58.846617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:58.848755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:58.861915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 927/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:53:59.042289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07577483-56b1-4c08-948c-9532771b7edb None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:59.045908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-07577483-56b1-4c08-948c-9532771b7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:53:59.071480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07577483-56b1-4c08-948c-9532771b7edb None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:59.081107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:59.082601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:59.083202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:59.083812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] PATH_INFO: `/v3/projects/129f5ff9fab74eaebfc211cf7c81ab0e` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:59.086229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=129f5ff9fab74eaebfc211cf7c81ab0e)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:59.089415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:59.207333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129f5ff9fab74eaebfc211cf7c81ab0e', 'request_id': 'req-b213d690-b778-4285-9a06-1fe7007aeebd'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.240029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129f5ff9fab74eaebfc211cf7c81ab0e', 'request_id': 'req-b213d690-b778-4285-9a06-1fe7007aeebd'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.260370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129f5ff9fab74eaebfc211cf7c81ab0e', 'request_id': 'req-b213d690-b778-4285-9a06-1fe7007aeebd'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.287672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] The token cache is being invalidate because project 129f5ff9fab74eaebfc211cf7c81ab0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:53:59.288284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.289389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.296463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:59.302328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 932/1859] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:59 2022] DELETE /identity/v3/projects/129f5ff9fab74eaebfc211cf7c81ab0e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:59.311606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2768c-95a0-4a52-b5b0-33bb686be343 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:53:59.433742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:53:59.434737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:53:59.435211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:53:59.435712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] PATH_INFO: `/v3/users/215419b9dd9947c98f7f9b7129a31e98` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:53:59.437616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215419b9dd9947c98f7f9b7129a31e98)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:53:59.440234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:53:59.442869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:53:59.533776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215419b9dd9947c98f7f9b7129a31e98'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.534951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215419b9dd9947c98f7f9b7129a31e98'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.536118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215419b9dd9947c98f7f9b7129a31e98'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.548608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215419b9dd9947c98f7f9b7129a31e98'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.578663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215419b9dd9947c98f7f9b7129a31e98'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:53:59.643328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 928/1860] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:53:59 2022] DELETE /identity/v3/users/215419b9dd9947c98f7f9b7129a31e98 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:53:59.984334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:53:59.989675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:00.020093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d690-b778-4285-9a06-1fe7007aeebd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:00.106066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:00.106613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.106891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.107163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] PATH_INFO: `/v3/projects/20b8940301a941f683e94d70bd31ae60` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.108400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b8940301a941f683e94d70bd31ae60)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:00.110379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:00.182015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b8940301a941f683e94d70bd31ae60', 'request_id': 'req-dd76a4b5-2b46-482d-9371-6c641f52af95'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.190561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b8940301a941f683e94d70bd31ae60', 'request_id': 'req-dd76a4b5-2b46-482d-9371-6c641f52af95'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.198424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:00.201007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b8940301a941f683e94d70bd31ae60', 'request_id': 'req-dd76a4b5-2b46-482d-9371-6c641f52af95'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.203533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:00.207296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:00.229280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de62225-a85a-43dc-939d-0d04067163b1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:00.256921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] The token cache is being invalidate because project 20b8940301a941f683e94d70bd31ae60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:00.257376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.261645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.267452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 933/1861] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:53:59 2022] DELETE /identity/v3/projects/20b8940301a941f683e94d70bd31ae60 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:00.276190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:00.283992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:00.433147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:00.435725 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.436331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.439539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] PATH_INFO: `/v3/projects/770acc5d03d84611b4e49a4b18819c29` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.444847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=770acc5d03d84611b4e49a4b18819c29)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:00.447528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:00.497123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a4b5-2b46-482d-9371-6c641f52af95 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:00.514288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770acc5d03d84611b4e49a4b18819c29', 'request_id': 'req-3c6f1b20-135f-410e-aec1-eb691915e7af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.524217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770acc5d03d84611b4e49a4b18819c29', 'request_id': 'req-3c6f1b20-135f-410e-aec1-eb691915e7af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.542407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770acc5d03d84611b4e49a4b18819c29', 'request_id': 'req-3c6f1b20-135f-410e-aec1-eb691915e7af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.576779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] The token cache is being invalidate because project 770acc5d03d84611b4e49a4b18819c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:00.577668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.580416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3c6f1b20-135f-410e-aec1-eb691915e7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.594123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 929/1862] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:00 2022] DELETE /identity/v3/projects/770acc5d03d84611b4e49a4b18819c29 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:00.594369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:00.594369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.594369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.594369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] PATH_INFO: `/v3/users/2812410eefe3487e8eb5edb47b118e18` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.594687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d20753-c515-45b0-a175-288c80148d5a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.594687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d20753-c515-45b0-a175-288c80148d5a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.594687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d20753-c515-45b0-a175-288c80148d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.595246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2812410eefe3487e8eb5edb47b118e18)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:00.599741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:00.631990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a0d20753-c515-45b0-a175-288c80148d5a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:00.634163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a0d20753-c515-45b0-a175-288c80148d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:00.649330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 930/1863] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:00.662745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.663249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.663759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.677269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:00.699164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2812410eefe3487e8eb5edb47b118e18'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.701547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2812410eefe3487e8eb5edb47b118e18'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.709364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:00.712525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:00.715615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2812410eefe3487e8eb5edb47b118e18'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.730719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2812410eefe3487e8eb5edb47b118e18'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.734158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 931/1864] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:00.735981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:00.750046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2812410eefe3487e8eb5edb47b118e18'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:00.757754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680cf4-c651-41a3-afb7-bbc6374abf50 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:00.765117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 934/1865] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:00 2022] DELETE /identity/v3/users/2812410eefe3487e8eb5edb47b118e18 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:00.773689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:00.775848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.777325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.777953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.779399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:00.780438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:00.784589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:00.816607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 932/1866] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:00.817359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:00.854169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:00.951255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d762a-3fc4-4407-ba75-75cfa28f5ecb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:00.965993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:00.966928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:00.967414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:00.967818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:00.968920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:00.974032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.031290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 935/1867] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:00 2022] POST /identity/v3/projects => generated 401 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:01.034056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.046204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de65a-10a3-4267-9e13-d7a174952255 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.155526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.156527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.156972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.157366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.158158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.160700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.194164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 933/1868] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:01 2022] POST /identity/v3/users => generated 301 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:01.195512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.207744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aff0c-63c1-4a26-9e89-2fbde3b0ab51 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.214300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.215125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.215530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.215882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.216632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.218887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.237273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 936/1869] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:01.238849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.250480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168c7bb-faa6-4d28-bbb7-00992ad712a9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.255853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.256486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.256981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.257257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.257977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.259942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.281739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 934/1870] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:01.287393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.297877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.299250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566ec7-98c1-4c80-b07e-5acd5ea66308 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.300205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:01.311236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.311961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.312313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.312579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] PATH_INFO: `/v3/projects/99a8c94bd9d04a34bfa2d693c6cdec58/users/3f48cf3fea7e4d5aa97f57e539884143/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.323523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e430f-a0d4-4bfe-9508-d9a0d0c55240 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.331977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.333204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.333657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.334143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] PATH_INFO: `/v3/projects/e9cf28f01e224b218cec4f3b6581489f` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.334899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a8c94bd9d04a34bfa2d693c6cdec58, user_id=3f48cf3fea7e4d5aa97f57e539884143, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.336071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cf28f01e224b218cec4f3b6581489f)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.337882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a2a66-140f-4c7d-adb8-2471332e6b7d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.338452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.358063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 937/1871] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:01 2022] PUT /identity/v3/projects/99a8c94bd9d04a34bfa2d693c6cdec58/users/3f48cf3fea7e4d5aa97f57e539884143/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:01.399382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.402091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.404279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.501337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cf28f01e224b218cec4f3b6581489f', 'request_id': 'req-95d621cd-fb18-4fc7-beff-aa780b7084ee'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:01.520880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cf28f01e224b218cec4f3b6581489f', 'request_id': 'req-95d621cd-fb18-4fc7-beff-aa780b7084ee'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:01.532615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] MFA Rules not processed for user `3f48cf3fea7e4d5aa97f57e539884143`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:01.534883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:01.541716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cf28f01e224b218cec4f3b6581489f', 'request_id': 'req-95d621cd-fb18-4fc7-beff-aa780b7084ee'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:01.563025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] The token cache is being invalidate because project e9cf28f01e224b218cec4f3b6581489f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:01.563697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:01.565708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-95d621cd-fb18-4fc7-beff-aa780b7084ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:01.571155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 935/1872] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:01 2022] DELETE /identity/v3/projects/e9cf28f01e224b218cec4f3b6581489f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:01.583955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2d53-6313-4116-9a24-15905d6c9699 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.585761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2d53-6313-4116-9a24-15905d6c9699 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.589643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2d53-6313-4116-9a24-15905d6c9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.632695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a96f2d53-6313-4116-9a24-15905d6c9699 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:01.634174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a96f2d53-6313-4116-9a24-15905d6c9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:01.733461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:01.775561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 936/1873] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:01.787895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.788347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.788674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.806215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 938/1874] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:01.822801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.825526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:01.865205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bea0ac-de91-41dd-87cb-79c91f3f1c6d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.868196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:01.870491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:01.886688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.887764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.888253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.888726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.889895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.893543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:01.918267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 937/1875] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:01.924351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:01.942785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54de9f-e529-4cf9-bb04-b0fdf7d0567d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:01.958389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:01.959467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:01.960132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:01.960422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:01.961365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:01.962383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:01.966045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.003707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 939/1876] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:01 2022] POST /identity/v3/projects => generated 401 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:02.008465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 938/1877] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:54:02.009386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.013166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.044745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aaf81d-d9d6-413d-993d-41457b463d91 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.121274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:02.197627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac108a0d-cef8-4d7e-a666-768e2f22e1a1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.233195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.235174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.235174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.235475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.236132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.238551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.284180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 940/1878] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:02 2022] POST /identity/v3/users => generated 307 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:02.289393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.311924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88730b24-3497-43ea-961f-6fb215d72215 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.312241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.313264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.313766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.315829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.317071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.320659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.327528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.328478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.328478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.328749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.329409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.331313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.362076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 941/1879] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:02.363317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.387234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d83626-4473-4261-acff-be5221a4d36f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.394161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.395476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.396004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.396497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] PATH_INFO: `/v3/projects/df0180dab517412d9a742403042e9c43/users/8154db1dc22947f7bbee6d02dae849ec/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.441489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0180dab517412d9a742403042e9c43, user_id=8154db1dc22947f7bbee6d02dae849ec, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.444583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.462115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 939/1880] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:02 2022] POST /identity/v3/projects => generated 377 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:02.464028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.472761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 942/1881] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:02 2022] PUT /identity/v3/projects/df0180dab517412d9a742403042e9c43/users/8154db1dc22947f7bbee6d02dae849ec/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:02.477195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.762388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a4c52-63a0-4705-bbbd-0797ebef59a0 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.778798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.779829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.780685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.781157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.782091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.787681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.806989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661296d5-d3ae-45c7-8d3d-a5fdb07acb1c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.807608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 943/1882] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:02.811999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.825418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.826003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.826358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.826779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.827392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.829557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.830805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9191-b84a-4f9c-b5f3-3954ccb38ffd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.850304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.852697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.852988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.855845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] PATH_INFO: `/v3/projects/df0180dab517412d9a742403042e9c43/users/8154db1dc22947f7bbee6d02dae849ec/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.864082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0180dab517412d9a742403042e9c43, user_id=8154db1dc22947f7bbee6d02dae849ec, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:02.866361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 940/1883] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:02 2022] POST /identity/v3/users => generated 289 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:02.867496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:02.867996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43dfa4-e48e-449c-86c2-655e8daaeee0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:02.884676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 944/1884] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:02 2022] PUT /identity/v3/projects/df0180dab517412d9a742403042e9c43/users/8154db1dc22947f7bbee6d02dae849ec/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:02.893314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289f574-6c14-486c-90ad-c85471d23b2e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:02.896764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.897053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.897323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:02.937992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] MFA Rules not processed for user `8154db1dc22947f7bbee6d02dae849ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:02.939508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:02.963246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:02.996923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:02.998925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:02.999544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:02.999901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.000936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.003187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.019384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 941/1885] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:03.020475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.028244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:03.033772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d99b2f-7048-4c00-8ee2-92ddfb32a505 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.039128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.039839 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.040146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.040448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.041402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 945/1886] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:03.043598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.045476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.053601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.059809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:03.068192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 942/1887] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:03.069952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.084419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73281b-2ecd-4f8f-90ed-645862bfb54c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.090741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.091914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.093037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.093697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] PATH_INFO: `/v3/projects/98d8013f05e54a23a4ffe2d827ef2f48/users/39e394f38bf5489cad8d6cce94bc3218/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.116613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d8013f05e54a23a4ffe2d827ef2f48, user_id=39e394f38bf5489cad8d6cce94bc3218, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.119433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f26791-5ce7-4db8-b452-45b3be53a95e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.136273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a29fa-f312-44df-9cd9-9c1cf42ef418 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.139334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 943/1888] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:03 2022] PUT /identity/v3/projects/98d8013f05e54a23a4ffe2d827ef2f48/users/39e394f38bf5489cad8d6cce94bc3218/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:03.149282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.149918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.150343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.188132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] MFA Rules not processed for user `39e394f38bf5489cad8d6cce94bc3218`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:03.189599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:03.197606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.198311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.198586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.198851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.277048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 944/1889] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:03.293781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.305124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.306609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.330165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 946/1890] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:03 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:03.393817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254e72a-9913-414d-b3be-6bbe4b876688 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.399533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.400501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.400728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.401194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.401989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.404213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.416538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.426341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40673a5-687c-4f9b-8a91-fbab6d692dd5 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.431665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.432519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.432829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.433097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.445254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 945/1891] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:03 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:03.446482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.469279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:03.516507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.518451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.525906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:03.529741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b7f0c-f622-4d0e-8b33-0377c0603b60 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.533515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 947/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:03 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:03.547076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.548179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.548668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.549155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.550375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.554476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.596683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 946/1893] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:03 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:03.600736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.614913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aafcfe-a83e-47cc-91db-1ed4c987a257 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.621008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.621671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.621947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.622224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.622895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.624750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.637903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 948/1894] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:03.643699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.659672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734755-5750-475b-9dac-98d1413dc6c4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.667669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.669446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.669446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.669446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.669919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.671557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b44225-5d10-4e7e-8518-bd9ae9142b42 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.686737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 947/1895] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:03.691748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:03.706479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458e29-3be7-44ca-9379-08c0abf93904 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:03.716534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:03.717850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.718522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.721386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] PATH_INFO: `/v3/projects/0fb3438dd5df46c59e28414b4251b87f/users/3d22d6895ac84345aaf8fa2b9e0510a4/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.765421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb3438dd5df46c59e28414b4251b87f, user_id=3d22d6895ac84345aaf8fa2b9e0510a4, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:03.772893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:03.814753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 949/1896] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:03 2022] PUT /identity/v3/projects/0fb3438dd5df46c59e28414b4251b87f/users/3d22d6895ac84345aaf8fa2b9e0510a4/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:03.834448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:03.837487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:03.838653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:03.909228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] MFA Rules not processed for user `3d22d6895ac84345aaf8fa2b9e0510a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:03.911026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:04.037188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 948/1897] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:04.042756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.228689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282e9bc-655f-47db-ba3c-05ede7c34966 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.242658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.243748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.244412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.244872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.248204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.252413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.330657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 950/1898] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:04 2022] POST /identity/v3/projects => generated 375 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:04.335449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.348480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577afb-f4ef-432f-9a52-a65f6da1b605 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.443800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.444579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.445050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.445415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.446204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.449548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.491274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 949/1899] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:04 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:04.496472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.518273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840969cf-67d9-43fd-a35e-8db8da7292e9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.526253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.527281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.527762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.528194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.529316 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.532661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.556552 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 951/1900] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:04.559123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.578066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e0708-1908-41f1-9b48-1bb4c141b085 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.583264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.583822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.584101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.584376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] PATH_INFO: `/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618/users/ffb6aa0ec0574b918a7589ee1241dd64/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.598990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:04.603955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac7b7f8a2e34fb79a4121c745b12618, user_id=ffb6aa0ec0574b918a7589ee1241dd64, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.608088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.634449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 950/1901] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:04 2022] PUT /identity/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618/users/ffb6aa0ec0574b918a7589ee1241dd64/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:04.638544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.657937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b92b2a-3d73-4146-9784-3b267294573c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.756126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.756767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.757047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.757344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.757982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.759909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d488b6e-f758-4248-aeed-f3c28e5a6195 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.773348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 952/1902] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:04.778051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:04.790860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d5b49-790d-4240-887a-738f3e1d8776 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:04.798462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:04.799206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.799500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.799777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] PATH_INFO: `/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618/users/ffb6aa0ec0574b918a7589ee1241dd64/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.802197 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac7b7f8a2e34fb79a4121c745b12618, user_id=ffb6aa0ec0574b918a7589ee1241dd64, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:04.805127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:04.827258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 951/1903] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:04 2022] PUT /identity/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618/users/ffb6aa0ec0574b918a7589ee1241dd64/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:04.831494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8896f52-f26d-491a-8ac7-b00ed1667bb2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:04.831861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8896f52-f26d-491a-8ac7-b00ed1667bb2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:04.832266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8896f52-f26d-491a-8ac7-b00ed1667bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:04.880201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b8896f52-f26d-491a-8ac7-b00ed1667bb2 None None] MFA Rules not processed for user `ffb6aa0ec0574b918a7589ee1241dd64`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:04.881656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b8896f52-f26d-491a-8ac7-b00ed1667bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:04.930174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:05.041802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 953/1904] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:05.060311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:05.155245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7600828-43d3-4777-b021-207f33a245bc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:05.159671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:05.160312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:05.160605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:05.160870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:05.230290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:05.231916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f44588-fc26-4838-9183-52ba81f0706e service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:05.243091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 952/1905] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:06.532330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:06.533746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:06.534511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:06.539279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 954/1906] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:54:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:54:06.556105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:06.557001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:06.557001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:06.610658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:06.612211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:06.849523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 953/1907] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:54:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:07.095936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:07.098208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:07.181808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e9cc-8415-4919-a0d0-b2ca66dd8fa6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:07.212799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:07.214989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:07.214989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:07.216037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:07.243346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:07.243346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:07.275772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 955/1908] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:07.331184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:07.342784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6556cd-93e9-4733-9569-38948e93f0bd None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:07.351784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:07.354480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:07.354889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:07.356937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:07.420376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:07.493385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:07.496230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:07.510772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 954/1909] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:08.947290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:08.949634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:08.973291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfd012-7b70-4bb7-b76a-eb6f063a2e4b service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:08.984744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:08.985802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:08.986269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:08.986774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:09.010136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:09.012141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7abc0-e959-49a8-8bac-b02992b7400d service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:09.036460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 956/1910] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:10.049414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:10.202985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62069-1f47-4247-8d69-d5b3cb412e81 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:10.213889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:10.215272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:10.215272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:10.215811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] PATH_INFO: `/v3/users/3f48cf3fea7e4d5aa97f57e539884143` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:10.218240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f48cf3fea7e4d5aa97f57e539884143)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:10.223537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:10.368190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f48cf3fea7e4d5aa97f57e539884143'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:10.369868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f48cf3fea7e4d5aa97f57e539884143'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:10.371483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f48cf3fea7e4d5aa97f57e539884143'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:10.385730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f48cf3fea7e4d5aa97f57e539884143'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:10.410450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f48cf3fea7e4d5aa97f57e539884143'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:10.427754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 955/1911] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:10 2022] DELETE /identity/v3/users/3f48cf3fea7e4d5aa97f57e539884143 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:10.434495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:10.435365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:10.435365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:10.467759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:10.469965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:10.575410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 957/1912] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:10.590191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:10.594105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:10.697262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:10.718599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496659f6-19b5-41e5-9bc7-9a707b618b8c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:10.723702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:10.724569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:10.725011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:10.725428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:10.738490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:10.740713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4f9fd-ae5b-4082-a7ec-7644e0515c31 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:10.759567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 956/1913] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:11.011530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.013833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.030775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc13b-4bdd-40bb-bb57-8bef8a682d5a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:11.036681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:11.037349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.037626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.037899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] PATH_INFO: `/v3/users/b36a2693c1134709888d14b284f45093` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.039755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36a2693c1134709888d14b284f45093)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:11.041767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:11.124945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36a2693c1134709888d14b284f45093'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.125810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36a2693c1134709888d14b284f45093'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.137073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36a2693c1134709888d14b284f45093'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.149715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36a2693c1134709888d14b284f45093'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.184875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36a2693c1134709888d14b284f45093'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.194758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 958/1914] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:11 2022] DELETE /identity/v3/users/b36a2693c1134709888d14b284f45093 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:11.199589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.200382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.200542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.230777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:11.232372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.352525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 957/1915] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:11.381915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.390173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.391463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.393325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.427623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:11.428414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080c370-b03e-40a1-9110-cd4a65943059 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:11.435666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:11.437367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.437897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.438504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] PATH_INFO: `/v3/users/0c9b4c6e52724f498513a888884a6a9c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.440540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9b4c6e52724f498513a888884a6a9c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:11.443614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:11.523697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f4fa3-2328-4be4-bb05-f85cc49b5740 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:11.538015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:11.539762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.540129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.540668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.560922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:11.564418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9b4c6e52724f498513a888884a6a9c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.565184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:11.565410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9b4c6e52724f498513a888884a6a9c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.566502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9b4c6e52724f498513a888884a6a9c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.580034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9b4c6e52724f498513a888884a6a9c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.590747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 959/1916] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:11.592859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.595637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.597917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1bb535cd-e41a-4e51-aa48-8c7ba1b49da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9b4c6e52724f498513a888884a6a9c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.625062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 958/1917] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:11 2022] DELETE /identity/v3/users/0c9b4c6e52724f498513a888884a6a9c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:11.628215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.628624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.629083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.638680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a7728-3b38-46c7-9c86-5855a36b3d81 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:11.654222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:11.655870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.754735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:11.756026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:11.756429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:11.757432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] PATH_INFO: `/v3/projects/99a8c94bd9d04a34bfa2d693c6cdec58` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:11.759842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a8c94bd9d04a34bfa2d693c6cdec58)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:11.762294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:11.782243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:11.811263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 959/1918] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:11.833765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.847561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:11.865463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a8c94bd9d04a34bfa2d693c6cdec58', 'request_id': 'req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.888720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a8c94bd9d04a34bfa2d693c6cdec58', 'request_id': 'req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.909580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a8c94bd9d04a34bfa2d693c6cdec58', 'request_id': 'req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.927119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] The token cache is being invalidate because project 99a8c94bd9d04a34bfa2d693c6cdec58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:11.927519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.928497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:11.930423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 960/1919] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:11 2022] DELETE /identity/v3/projects/99a8c94bd9d04a34bfa2d693c6cdec58 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:11.935286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:11.938049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:12.012739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:12.013091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1613c2-8583-4c80-978d-c1ca20ef8fb5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:12.056340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2596e9a-5fee-48b1-84ef-3a5066eef2fb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:12.066749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:12.067412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:12.067978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:12.067978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] PATH_INFO: `/v3/users/8154db1dc22947f7bbee6d02dae849ec` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:12.069145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8154db1dc22947f7bbee6d02dae849ec)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:12.073794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:12.196439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8154db1dc22947f7bbee6d02dae849ec'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.197145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8154db1dc22947f7bbee6d02dae849ec'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.211843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:12.212514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:12.212797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:12.213042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:12.217282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8154db1dc22947f7bbee6d02dae849ec'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.218213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:12.232572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8154db1dc22947f7bbee6d02dae849ec'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.261100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:12.265935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b70d0d-6cc4-410f-b367-ad8f3cd75db8 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:12.271823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8154db1dc22947f7bbee6d02dae849ec'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.294213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 961/1920] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:11 2022] DELETE /identity/v3/users/8154db1dc22947f7bbee6d02dae849ec => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:12.294213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:12.307170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:12.308489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 960/1921] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:12.443477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9a74f-1ed3-4c89-b061-e8110a1d1003 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:12.457300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:12.458164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:12.458865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:12.459290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] PATH_INFO: `/v3/projects/b248d3a73c8b4361b2375aafee2508d6` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:12.460958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b248d3a73c8b4361b2375aafee2508d6)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:12.463411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:12.563762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b248d3a73c8b4361b2375aafee2508d6', 'request_id': 'req-99fb087c-c764-4284-9343-3d5608ffffcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.577563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b248d3a73c8b4361b2375aafee2508d6', 'request_id': 'req-99fb087c-c764-4284-9343-3d5608ffffcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.587107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:12.591328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b248d3a73c8b4361b2375aafee2508d6', 'request_id': 'req-99fb087c-c764-4284-9343-3d5608ffffcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.621846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] The token cache is being invalidate because project b248d3a73c8b4361b2375aafee2508d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:12.622874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.624236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:12.627168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 962/1922] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:12 2022] DELETE /identity/v3/projects/b248d3a73c8b4361b2375aafee2508d6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:12.636741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:12.636741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:12.636741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:12.807081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:12.817338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:12.828788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:12.830411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:12.995756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb087c-c764-4284-9343-3d5608ffffcc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.005328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.006433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.007472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.007642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] PATH_INFO: `/v3/projects/33db7a46234c423fb07ab2111c9460df` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.010888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33db7a46234c423fb07ab2111c9460df)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.016355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:13.083808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 961/1923] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:13.083808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.083808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.098357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b7f49-e25b-4919-8fbb-6a3970df8d69 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.118318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.119171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.119526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.119803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] PATH_INFO: `/v3/projects/df0180dab517412d9a742403042e9c43` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.121084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0180dab517412d9a742403042e9c43)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.123364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:13.157833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33db7a46234c423fb07ab2111c9460df', 'request_id': 'req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.171835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33db7a46234c423fb07ab2111c9460df', 'request_id': 'req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.188238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33db7a46234c423fb07ab2111c9460df', 'request_id': 'req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.212134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] The token cache is being invalidate because project 33db7a46234c423fb07ab2111c9460df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:13.212541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.213400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-671fd6b9-faa2-4ff8-9fe9-4ce44056d01f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.216219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 963/1924] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:12 2022] DELETE /identity/v3/projects/33db7a46234c423fb07ab2111c9460df => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:13.226696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa59-adb0-40bf-831e-a206b0ab3c03 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.227162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa59-adb0-40bf-831e-a206b0ab3c03 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.227475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aaa59-adb0-40bf-831e-a206b0ab3c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.234938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0180dab517412d9a742403042e9c43', 'request_id': 'req-18e8c69f-9d62-4a3f-a81a-df35bf774849'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.237395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:13.246722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0180dab517412d9a742403042e9c43', 'request_id': 'req-18e8c69f-9d62-4a3f-a81a-df35bf774849'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.257682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0180dab517412d9a742403042e9c43', 'request_id': 'req-18e8c69f-9d62-4a3f-a81a-df35bf774849'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.267580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-af5aaa59-adb0-40bf-831e-a206b0ab3c03 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:13.269300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-af5aaa59-adb0-40bf-831e-a206b0ab3c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.273661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] The token cache is being invalidate because project df0180dab517412d9a742403042e9c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:13.274169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.274934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.276988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 962/1925] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:13 2022] DELETE /identity/v3/projects/df0180dab517412d9a742403042e9c43 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:13.277751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.281049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.390983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 964/1926] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:13.397296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.397582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.397849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.401383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8c69f-9d62-4a3f-a81a-df35bf774849 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.410668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.412071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.413424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.413720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] PATH_INFO: `/v3/users/11fda0a40de24305bc18a12bb4454288` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.417000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fda0a40de24305bc18a12bb4454288)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.420721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:13.445801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:13.447478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.461817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 965/1927] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:13.462742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.479168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550ceb-537e-4a35-89b7-310beb752eaf None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.492517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.493161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.493436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.494150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.494828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:13.495133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.498124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45196d-6e24-4781-857f-ba60a1ded3b2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:13.519183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:13.525810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 966/1928] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:13.528615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.529223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.530094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.576015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:13.578582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.591369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 967/1929] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:13.591369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.616571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6932c-739e-4c75-84b5-3221adcf52a3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.617342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fda0a40de24305bc18a12bb4454288'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.620680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fda0a40de24305bc18a12bb4454288'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.625157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fda0a40de24305bc18a12bb4454288'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.642762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fda0a40de24305bc18a12bb4454288'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.652306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.654459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.654459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.654459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.657950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.660036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:13.674830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fda0a40de24305bc18a12bb4454288'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:13.711932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 963/1930] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:13 2022] DELETE /identity/v3/users/11fda0a40de24305bc18a12bb4454288 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:13.713001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.714766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.758336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 968/1931] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:13 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:13.760456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:13.763825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:13.882221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:13.891763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e0ae0-91b5-4d39-bf2f-cc5c82deceba admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.942341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a835-d10d-45a1-84ec-992aa464c7a2 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:13.952812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:13.953674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:13.954110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:13.954550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] PATH_INFO: `/v3/users/39e394f38bf5489cad8d6cce94bc3218` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:13.957670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e394f38bf5489cad8d6cce94bc3218)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:13.962482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.079795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.080528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.080872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.084300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.086195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:14.086507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.094082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.114509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e394f38bf5489cad8d6cce94bc3218'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.117070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e394f38bf5489cad8d6cce94bc3218'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.122887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 964/1932] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:14.128816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.132162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.133477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e394f38bf5489cad8d6cce94bc3218'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.146220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e394f38bf5489cad8d6cce94bc3218'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.151148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aee37e-e8ca-4014-837f-d2920abd1e11 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.167117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.167805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.168080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.168384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.169681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.171640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aff0b-b59e-4b11-a8d1-9a6e095a3778 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.176624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e394f38bf5489cad8d6cce94bc3218'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.191213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 969/1933] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:13 2022] DELETE /identity/v3/users/39e394f38bf5489cad8d6cce94bc3218 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:14.192823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.199391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.203890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 965/1934] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/users => generated 288 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:14.205405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.205693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.205968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.232923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:14.234228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.351535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:14.353176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 966/1935] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:14.355696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.377360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70935b-3386-4d39-8041-89b5e8fddc88 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.386088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.387182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.391416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.392467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39085253-2c9f-4342-8818-b11ca79f85f8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.393606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.395038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.397176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.400635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.401904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.402312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.402678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.403782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.406342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.419447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 967/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:14.420668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.429204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.450370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f1f4-c2f5-44df-a55e-c5ee24387ec3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.455417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 970/1937] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:14.455417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.462328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.535149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.535837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.536111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.536380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] PATH_INFO: `/v3/projects/14e6cff0cad2468c81801a099f74b950` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.537647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e6cff0cad2468c81801a099f74b950)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.540114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.562956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:14.613296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d20e74-07c8-47d1-b163-e4e00edd767c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.627029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e6cff0cad2468c81801a099f74b950', 'request_id': 'req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.639206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e6cff0cad2468c81801a099f74b950', 'request_id': 'req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.653581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e6cff0cad2468c81801a099f74b950', 'request_id': 'req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.672265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] The token cache is being invalidate because project 14e6cff0cad2468c81801a099f74b950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:14.672872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.673875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:14.676242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 968/1938] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:14 2022] DELETE /identity/v3/projects/14e6cff0cad2468c81801a099f74b950 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:14.677979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.683611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.689044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.689682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.690009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.690280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.692305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.817582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e9c7-8c3e-4fa3-9bb7-cfc8c1f91c68 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.818003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.819496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.826489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.827597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.828111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.828759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.830028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.830809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 971/1939] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:14.833810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d810-4b3b-4e13-acab-78b41a576c94 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.840195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.849614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.856029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 969/1940] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:14.858532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.859020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.859537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.865349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965890-f7f1-48da-a396-1a0e1495c1d3 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.871595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.873046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.873046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.873490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.874594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.878870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.902339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:14.906415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:14.914438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 972/1941] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/users => generated 291 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:14.914438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.927672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 970/1942] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:14.929157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:14.935118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486bea-9897-40ba-9245-9017636b8e06 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.944303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.945007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.945337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.945617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] PATH_INFO: `/v3/projects/be4de44b5992457fa3b199238507323e/users/fc391ad6edf447cfbed98cb324f104ef/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.963503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d50ef-3804-41b0-b3bf-fdcf66f9c83d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:14.966594 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4de44b5992457fa3b199238507323e, user_id=fc391ad6edf447cfbed98cb324f104ef, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.969994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1f3df-fd07-417f-8883-fcef4463c538 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.974555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:14.978157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:14.978881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:14.979657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:14.981349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:14.983481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:14.985032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2b2bc-0aba-4c31-a319-03bf31b7648d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:14.992467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 973/1943] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:14 2022] PUT /identity/v3/projects/be4de44b5992457fa3b199238507323e/users/fc391ad6edf447cfbed98cb324f104ef/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:14.999517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.000180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.002337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.013626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 971/1944] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:15.016474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.016955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.017410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.054129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:15.055345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:15.115508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] MFA Rules not processed for user `fc391ad6edf447cfbed98cb324f104ef`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:15.117808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:15.215881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 974/1945] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:15.217705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.235915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 972/1946] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:15.236624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.243610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea25eb-52a7-4cb1-9b14-f1bd6de979f1 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.252076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.253228 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.253747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.254317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.255751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.257189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84427fb-30a3-470c-8736-3240b634d439 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.259657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.265403 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.266387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.266834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.267312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.268152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:15.268627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.270687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.274158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:15.283169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 975/1947] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:15.283169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.286290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:15.293377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 973/1948] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:15.293377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.314130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb770f-5c23-4b36-b0d2-ce2293543e73 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.324985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.326421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.327106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.327626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] PATH_INFO: `/v3/projects/a44c495734e949aaad99f210762fcb98/users/afed5bf79f9e4a27850be3d87fabfdf2/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.356943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44c495734e949aaad99f210762fcb98, user_id=afed5bf79f9e4a27850be3d87fabfdf2, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.364614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.386484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 974/1949] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:15 2022] PUT /identity/v3/projects/a44c495734e949aaad99f210762fcb98/users/afed5bf79f9e4a27850be3d87fabfdf2/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:15.389225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e4d7-ad32-409d-aac1-22a3fad7455a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.392292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.489781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.490727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.491182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.491695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.510665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd370b-bec9-4a9e-a7bd-164e3a2de636 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.519279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.520949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.521660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.522405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.524271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.529186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699db20f-f31c-43ab-bbbe-15606c96bd2b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.630539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 975/1950] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:15 2022] POST /identity/v3/projects => generated 381 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:15.633159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.633894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.634553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.650302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.653293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.676539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:15.694609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 976/1951] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:15.696957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.702140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:15.737580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] MFA Rules not processed for user `afed5bf79f9e4a27850be3d87fabfdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:15.739824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:15.750043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3310ed-ccbc-49af-b5e8-04f7b212b9e7 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:15.936543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:15.937163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:15.937504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:15.937827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] PATH_INFO: `/v3/projects/98d8013f05e54a23a4ffe2d827ef2f48` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:15.942092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d8013f05e54a23a4ffe2d827ef2f48)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:15.943586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 976/1952] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:15.944643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:15.944998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:15.992440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a8cf2-8b45-4ef9-898c-71748f8ae08a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.005782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.009377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.010156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.010650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.012689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.016962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.059712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d8013f05e54a23a4ffe2d827ef2f48', 'request_id': 'req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.076703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d8013f05e54a23a4ffe2d827ef2f48', 'request_id': 'req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.079262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 977/1953] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:15 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:16.083942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.093518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d8013f05e54a23a4ffe2d827ef2f48', 'request_id': 'req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.111919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d7330-1585-405c-9204-50c131b921a7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.121608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] The token cache is being invalidate because project 98d8013f05e54a23a4ffe2d827ef2f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:16.121890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.122672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.129109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 977/1954] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:15 2022] DELETE /identity/v3/projects/98d8013f05e54a23a4ffe2d827ef2f48 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:16.129109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.129109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:16.219338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:16.290966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2d05-4d00-4610-b774-c23a6bfe7ee3 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.305896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.307037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.307566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.308055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.310380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.314876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.319006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.321111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.321111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.321368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.326586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.332422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.347584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 978/1955] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:16.347584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.349226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:16.369966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff0271-227b-4796-9b9d-07e165f45def admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.380302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.381573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.383389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.384009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] PATH_INFO: `/v3/users/3d22d6895ac84345aaf8fa2b9e0510a4` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.387542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d22d6895ac84345aaf8fa2b9e0510a4)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.392550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.465659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 978/1956] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:16.468454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.508808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:16.583818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d22d6895ac84345aaf8fa2b9e0510a4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.585103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d22d6895ac84345aaf8fa2b9e0510a4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.593154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9b5c-08e7-4aaf-9ecb-f7aa03a6fcb6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.599281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.600031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.600319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.600715 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.601425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.602042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d22d6895ac84345aaf8fa2b9e0510a4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.603119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.618030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 979/1957] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:16.619001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.649355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d22d6895ac84345aaf8fa2b9e0510a4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.650936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:16.680578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5dcc5-f185-4311-8e29-5054113a9a30 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.699779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.700840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.701008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.701714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.702672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.707467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.708399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d22d6895ac84345aaf8fa2b9e0510a4'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:16.729002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 979/1958] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:16 2022] DELETE /identity/v3/users/3d22d6895ac84345aaf8fa2b9e0510a4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:16.729002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.732597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:16.799997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 980/1959] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:16 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:16.800994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:16.815492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993deef2-7ea1-4d26-bf4a-7e93b4c87ed9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.905670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044056-2963-4f3a-897e-6fe45e6dd325 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:16.916457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.917729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.918208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.918669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] PATH_INFO: `/v3/projects/de4696abe6884fc8af9d48e58a19be7a/users/cac271e4b1194566a0ac2d1e669fe9ed/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.955613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de4696abe6884fc8af9d48e58a19be7a, user_id=cac271e4b1194566a0ac2d1e669fe9ed, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.958153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:16.958817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:16.959098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:16.959660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:16.959867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9add8b-fa72-4b6a-8e6d-7fbee4cffb7b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.960354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:16.962054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:16.996423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 981/1960] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:17.007750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.009810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 980/1961] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:16 2022] PUT /identity/v3/projects/de4696abe6884fc8af9d48e58a19be7a/users/cac271e4b1194566a0ac2d1e669fe9ed/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:17.013056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.015553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.016592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.025028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84faf0-b784-4cbe-94ce-4b89d45d3f89 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.111662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] MFA Rules not processed for user `cac271e4b1194566a0ac2d1e669fe9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:17.112856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:17.117122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:17.211553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.212280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.214889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.215274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] PATH_INFO: `/v3/projects/bc349397d8fe4d829493fc44d4fd478d/users/414eb727b79b403380752cc4c8eaa575/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.257906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc349397d8fe4d829493fc44d4fd478d, user_id=414eb727b79b403380752cc4c8eaa575, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.268228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:17.299757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 982/1962] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:16 2022] PUT /identity/v3/projects/bc349397d8fe4d829493fc44d4fd478d/users/414eb727b79b403380752cc4c8eaa575/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:17.305534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.309533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 981/1963] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:17.312485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.341040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d2612-2d46-46f9-ba47-ed86df410a45 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.359266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:17.462667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.463348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.463577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d6b8-c953-4a0b-84fc-a337fccbf145 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.463784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.464078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.464736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.466763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:17.476891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.478368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.478866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.479368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.480260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.482296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:17.504233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 983/1964] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:17.509911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.534058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 982/1965] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:17 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:17.534291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283f72-e4a6-443d-89ac-ff00b2c94b86 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.537086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.731914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.733343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.733724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.734696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] PATH_INFO: `/v3/projects/bc349397d8fe4d829493fc44d4fd478d/users/414eb727b79b403380752cc4c8eaa575/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.743690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc349397d8fe4d829493fc44d4fd478d, user_id=414eb727b79b403380752cc4c8eaa575, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.750411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:17.756509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3793e-592f-4117-9be9-353ef5d26c8d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.772620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.777001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.777352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.777678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.781562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.786774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 984/1966] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:17 2022] PUT /identity/v3/projects/bc349397d8fe4d829493fc44d4fd478d/users/414eb727b79b403380752cc4c8eaa575/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:17.787835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:17.788997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fea60-b6aa-4177-a5fc-7612e5257a06 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:17.802332 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:17.854087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 983/1967] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:17 2022] POST /identity/v3/users => generated 297 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:17.854087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.854087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.854807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.887203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:17.915691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] MFA Rules not processed for user `414eb727b79b403380752cc4c8eaa575`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:17.917080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:17.942428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90c445-48ac-4dc0-947d-bfd50b5bd70c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:17.961951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:17.963172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:17.963701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:17.964748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] PATH_INFO: `/v3/projects/0fb3438dd5df46c59e28414b4251b87f` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:17.967585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb3438dd5df46c59e28414b4251b87f)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:17.979609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.027350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 984/1968] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:18.027350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.051883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259700c2-bcd3-4a19-8f71-e2ac6503d21f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.061619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51334637-c009-4c25-8180-823940ee267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.062519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51334637-c009-4c25-8180-823940ee267f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.062519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51334637-c009-4c25-8180-823940ee267f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.063056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51334637-c009-4c25-8180-823940ee267f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.064051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51334637-c009-4c25-8180-823940ee267f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.065892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51334637-c009-4c25-8180-823940ee267f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.072956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:18.079401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 985/1969] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:18.083275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51334637-c009-4c25-8180-823940ee267f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.084586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-51334637-c009-4c25-8180-823940ee267f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.101949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb3438dd5df46c59e28414b4251b87f', 'request_id': 'req-f918da7c-ec02-4c8c-8ff5-799623f21609'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.115445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb3438dd5df46c59e28414b4251b87f', 'request_id': 'req-f918da7c-ec02-4c8c-8ff5-799623f21609'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.126446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb3438dd5df46c59e28414b4251b87f', 'request_id': 'req-f918da7c-ec02-4c8c-8ff5-799623f21609'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.143118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] The token cache is being invalidate because project 0fb3438dd5df46c59e28414b4251b87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:18.143450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.144204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.149452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 985/1970] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:17 2022] DELETE /identity/v3/projects/0fb3438dd5df46c59e28414b4251b87f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:18.149452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.151305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.183374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51334637-c009-4c25-8180-823940ee267f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.252981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.253193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918da7c-ec02-4c8c-8ff5-799623f21609 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.253896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.255056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.255402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.257274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.259468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.260240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.260670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.261051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] PATH_INFO: `/v3/projects/0babeac690fa4262b997bbe5520a5828/users/9a9f7e31ee6f4295aec0c43d141028d7/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.280208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0babeac690fa4262b997bbe5520a5828, user_id=9a9f7e31ee6f4295aec0c43d141028d7, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.283085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.302643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 986/1971] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:18 2022] PUT /identity/v3/projects/0babeac690fa4262b997bbe5520a5828/users/9a9f7e31ee6f4295aec0c43d141028d7/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:18.303944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.306333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.430809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.433165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.443848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:18.461492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 986/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:18.464855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ed9a-196d-4d18-ac1c-dd82bb17ce5a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.465734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.491835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.492899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.492899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.493207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] PATH_INFO: `/v3/users/ffb6aa0ec0574b918a7589ee1241dd64` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.496648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb6aa0ec0574b918a7589ee1241dd64)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.498615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c233-c4eb-4271-994e-c7a8c9b4b5dd service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.498765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.514971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.514971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.515851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.516102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.517099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.525175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.597775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 987/1973] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:18.603595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.611542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb6aa0ec0574b918a7589ee1241dd64'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.612533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb6aa0ec0574b918a7589ee1241dd64'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.615044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb6aa0ec0574b918a7589ee1241dd64'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.626890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010d78-8a11-4528-87b3-c47111d52735 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.634689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.635047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.635480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.636332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] PATH_INFO: `/v3/projects/0babeac690fa4262b997bbe5520a5828/users/9a9f7e31ee6f4295aec0c43d141028d7/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.639698 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0babeac690fa4262b997bbe5520a5828, user_id=9a9f7e31ee6f4295aec0c43d141028d7, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:18.643850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e13a-9726-41b7-9f57-365d6492684a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:18.651869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb6aa0ec0574b918a7589ee1241dd64'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.679835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 988/1974] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:18 2022] PUT /identity/v3/projects/0babeac690fa4262b997bbe5520a5828/users/9a9f7e31ee6f4295aec0c43d141028d7/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:18.681714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.681996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.682305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.722669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb6aa0ec0574b918a7589ee1241dd64'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:18.760329 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 987/1975] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:18 2022] DELETE /identity/v3/users/ffb6aa0ec0574b918a7589ee1241dd64 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:18.763667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:18.767891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] MFA Rules not processed for user `9a9f7e31ee6f4295aec0c43d141028d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:18.771521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.778089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.896290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:18.921940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850d90-fb68-45dc-a346-bc69838f7f44 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:18.932577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:18.935464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:18.937456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:18.937828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:18.953405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:18.998073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 989/1976] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:19.020294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.022989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:19.073680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d17cf-ffc5-457b-bc5c-69de492ac00f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:19.079256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:19.079924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.080207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.080481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.099135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.103274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.116771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.120925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.147450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 988/1977] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:19.163541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 990/1978] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:19.236994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.238775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:19.302043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:19.325541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.338280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870db9-d96d-4cc0-bcc5-3e3861f8ce4e service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:19.343400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:19.343567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.343712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.343984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.345998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:19.349743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7beb0-7249-48bc-9430-e8e9136b9065 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:19.355691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:19.358599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.361762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.362423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] PATH_INFO: `/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.367449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac7b7f8a2e34fb79a4121c745b12618)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.373095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.458671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.461696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2194c-2256-4188-8830-fb75b5036604 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.481634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 991/1979] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:19.489343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac7b7f8a2e34fb79a4121c745b12618', 'request_id': 'req-510ce445-3cda-4154-92ad-feb90d812e85'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:19.498042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac7b7f8a2e34fb79a4121c745b12618', 'request_id': 'req-510ce445-3cda-4154-92ad-feb90d812e85'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:19.511374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac7b7f8a2e34fb79a4121c745b12618', 'request_id': 'req-510ce445-3cda-4154-92ad-feb90d812e85'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:19.533771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] The token cache is being invalidate because project 2ac7b7f8a2e34fb79a4121c745b12618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:19.534372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:19.536635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-510ce445-3cda-4154-92ad-feb90d812e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:19.541007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 989/1980] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:19 2022] DELETE /identity/v3/projects/2ac7b7f8a2e34fb79a4121c745b12618 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:19.551939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.552884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.553940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.594962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:19.597535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:19.711270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 992/1981] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:19.714804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.715082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.715424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.745552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:19.747646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:19.763275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 990/1982] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:19.764695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.801750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071390a-1e6f-4054-86e0-343bb05fb128 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:19.827320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:19.827320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.827320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.827320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.828119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:19.828119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.828119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.870898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 993/1983] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:19.877352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.895187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a744f9f-f141-40b2-98cb-4eb00d13ea7d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:19.904152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:19.905810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:19.906343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:19.906949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:19.908206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:19.914899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:19.966491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 991/1984] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:19 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:19.969782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:19.980918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6dfd2-7c91-412f-91cf-22053084cba4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.043038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:20.072694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.077115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.079314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.080068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.080348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.080820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.081651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.083773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.107832 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f83cf7-9b58-422d-84fc-0d75587e4de7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.118735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.118735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.118735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.118735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] PATH_INFO: `/v3/users/fc391ad6edf447cfbed98cb324f104ef` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.119884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc391ad6edf447cfbed98cb324f104ef)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.120317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 994/1985] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:19 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:20.123071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.123749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.146038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4f0d-c6fc-4740-bb10-f09bd923c710 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.155746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.156721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.157188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.157619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.158968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.162245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.194477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 995/1986] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:20.196744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.211024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc391ad6edf447cfbed98cb324f104ef'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:20.212073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc391ad6edf447cfbed98cb324f104ef'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:20.213036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc391ad6edf447cfbed98cb324f104ef'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:20.215611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396cf2a-bc62-475d-b864-f30437ab39f2 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.224042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.224973 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.225521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.225910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.226898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.227069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc391ad6edf447cfbed98cb324f104ef'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:20.229775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.252365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53b32d6d-2f9e-4dfa-a14e-bb110ef85f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc391ad6edf447cfbed98cb324f104ef'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:20.262988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 996/1987] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:20.273673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.277582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.278921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 992/1988] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:20 2022] DELETE /identity/v3/users/fc391ad6edf447cfbed98cb324f104ef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:20.291757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.292420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.292809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.305801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced21f9-93fb-420c-a184-a13eb44363c7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.306018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:20.332065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:20.335617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.413033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.414551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.415303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.415814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] PATH_INFO: `/v3/projects/6c2e1cfab8324f15880a1ad2f5adea3d/users/8ba507ef2c514e4a99defc6df3320e74/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.457787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2e1cfab8324f15880a1ad2f5adea3d, user_id=8ba507ef2c514e4a99defc6df3320e74, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.464310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adefa-75e2-49e1-a048-f78d92cfa141 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.478769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 993/1989] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:20.493427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.496171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.500680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 997/1990] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:20 2022] PUT /identity/v3/projects/6c2e1cfab8324f15880a1ad2f5adea3d/users/8ba507ef2c514e4a99defc6df3320e74/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:20.524075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.524537 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.527777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.585155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] MFA Rules not processed for user `8ba507ef2c514e4a99defc6df3320e74`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:20.587063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.685650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c133644-518b-4fa4-9af9-f807a5c7732f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.716831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 998/1991] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:20.732042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:20.734633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:20.768686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:20.775877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.776520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.776793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.777056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.816894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dbc40-4311-4cee-8afd-b393a63e5fab None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:20.820905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:20.821663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:20.821949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:20.822219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:20.832304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.834359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.853651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 999/1992] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:20.876350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:20.880137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a70f9-7b02-459a-b0b5-298c86e1d954 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:20.915391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 994/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:21.389089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:21.391770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:21.446608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527acd-3739-4343-ad5e-1253ac48a1a7 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:21.464806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:21.465812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:21.466322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:21.466802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] PATH_INFO: `/v3/projects/be4de44b5992457fa3b199238507323e` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:21.470180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be4de44b5992457fa3b199238507323e)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:21.474003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:21.620955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be4de44b5992457fa3b199238507323e', 'request_id': 'req-e155e6c0-c5a2-4204-af00-d4191d86cc71'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:21.637323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be4de44b5992457fa3b199238507323e', 'request_id': 'req-e155e6c0-c5a2-4204-af00-d4191d86cc71'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:21.653224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be4de44b5992457fa3b199238507323e', 'request_id': 'req-e155e6c0-c5a2-4204-af00-d4191d86cc71'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:21.710158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] The token cache is being invalidate because project be4de44b5992457fa3b199238507323e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:21.710487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:21.711695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e155e6c0-c5a2-4204-af00-d4191d86cc71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:21.718597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1000/1994] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:21 2022] DELETE /identity/v3/projects/be4de44b5992457fa3b199238507323e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:21.720764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:21.723570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:21.724317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:21.788168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:21.791077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:21.801025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:21.917736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 995/1995] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:21.933067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:21.933695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:21.934475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:21.972496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:21.974009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:21.983278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1001/1996] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:21.990136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.006868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175393-c395-4a25-a0f0-b122dd5ebe2b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.014633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.015551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.016017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.016272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.017411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:22.017726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.020087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.043355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 996/1997] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:54:22.044841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.057534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f72c46-e6ec-4abf-b0b1-1aea4d3d87cf None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.064048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.065554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.065727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.066185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.067615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.069945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.112623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1002/1998] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:22 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:22.119067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.137911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e39c7-3037-4885-a9cf-3d6f434d5d00 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.269569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.270946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.271456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.271935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.274496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.278495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.327071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 997/1999] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:22 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:22.332528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.345999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0104-8c44-4726-b820-341a2545beac admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.351699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.352233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.352511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.352784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.354179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.355928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.378541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1003/2000] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:22.379804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.392056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d42ae-0378-4da5-bb23-117f5ece4585 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.397316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.398111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.398396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.398671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.400568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.401747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b925021-fbeb-45db-955b-cabe4efd798a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.424040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 998/2001] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:22.429916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.449197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c5f6d-9a9c-4d23-ba08-e7a16c4dce9b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.460633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.461871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.462368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.462826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] PATH_INFO: `/v3/projects/350e5a298325458bb3657e02c7de30c2/users/1abc96453ab2428ab59ca3935666bfcc/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.497404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350e5a298325458bb3657e02c7de30c2, user_id=1abc96453ab2428ab59ca3935666bfcc, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.502950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:22.563278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1004/2002] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:22 2022] PUT /identity/v3/projects/350e5a298325458bb3657e02c7de30c2/users/1abc96453ab2428ab59ca3935666bfcc/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:22.586976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.587075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.587377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.641308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] MFA Rules not processed for user `1abc96453ab2428ab59ca3935666bfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:22.642828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:22.738988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 999/2003] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:22.748115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.750254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:22.783083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:22.788863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:22.798721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:22.862059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:22.950187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4f8b0-e640-4ea7-9c72-f61db6fb9693 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:22.960831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:22.965024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:22.965311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:22.965588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:22.983952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:22.985603 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f80a98-d3e2-4097-8705-2eb149b01a6e service glance] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:23.004786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1005/2004] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:23.014632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20473056-364e-4eae-8dd8-2d7d40cbe57e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:23.020805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:23.022446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:23.022900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:23.023519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] PATH_INFO: `/v3/users/afed5bf79f9e4a27850be3d87fabfdf2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:23.025409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afed5bf79f9e4a27850be3d87fabfdf2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:23.028387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:23.143569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afed5bf79f9e4a27850be3d87fabfdf2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:23.145630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afed5bf79f9e4a27850be3d87fabfdf2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:23.146653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afed5bf79f9e4a27850be3d87fabfdf2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:23.163710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afed5bf79f9e4a27850be3d87fabfdf2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:23.193953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afed5bf79f9e4a27850be3d87fabfdf2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:23.228766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1000/2005] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:22 2022] DELETE /identity/v3/users/afed5bf79f9e4a27850be3d87fabfdf2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:23.230825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:23.231108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:23.231407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:23.273541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:23.279104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:23.432781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1006/2006] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:23.506140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:23.507692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:23.597820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9330db-50c1-40ca-a82f-7b761e884f77 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:23.601952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:23.602840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:23.603126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:23.603454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:23.615712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:23.617558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7c24-6b8b-457e-9175-4f1af684c421 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:23.634043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1001/2007] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:24.157107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:24.160123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:24.181223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2e924-7ab1-48fe-af84-23b5820b4377 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:24.189248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:24.190393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:24.190850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:24.191459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] PATH_INFO: `/v3/users/8ba507ef2c514e4a99defc6df3320e74` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:24.193683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba507ef2c514e4a99defc6df3320e74)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:24.197317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:24.303007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba507ef2c514e4a99defc6df3320e74'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:24.305206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba507ef2c514e4a99defc6df3320e74'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:24.323601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba507ef2c514e4a99defc6df3320e74'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:24.338151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba507ef2c514e4a99defc6df3320e74'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:24.367058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba507ef2c514e4a99defc6df3320e74'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:24.379875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1007/2008] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:24 2022] DELETE /identity/v3/users/8ba507ef2c514e4a99defc6df3320e74 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:24.385065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:24.385903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:24.385903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:24.410367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:24.410367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:24.438974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:24.506617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1002/2009] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:24.552308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:24.554399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:24.627826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:24.652232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aff4c-5a33-4930-ad5a-5eb2ef5151e4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:24.656856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:24.657292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:24.657588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:24.657857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:24.669658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:24.671518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:24.684907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1008/2010] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:24.895276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:24.896939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:24.912229 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4a739-8731-4302-a7cc-aedcc530d3fd None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:24.918127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:24.918904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:24.919184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:24.919516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] PATH_INFO: `/v3/projects/a44c495734e949aaad99f210762fcb98` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:24.921646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44c495734e949aaad99f210762fcb98)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:24.923890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:25.030770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44c495734e949aaad99f210762fcb98', 'request_id': 'req-7fe502ce-2688-44c0-bdb5-2d376a389291'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.060291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44c495734e949aaad99f210762fcb98', 'request_id': 'req-7fe502ce-2688-44c0-bdb5-2d376a389291'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.082767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44c495734e949aaad99f210762fcb98', 'request_id': 'req-7fe502ce-2688-44c0-bdb5-2d376a389291'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.125324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] The token cache is being invalidate because project a44c495734e949aaad99f210762fcb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:25.125856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.126636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.128875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1003/2011] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:24 2022] DELETE /identity/v3/projects/a44c495734e949aaad99f210762fcb98 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:25.132981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:25.146031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cf3c8-ae7f-43ca-8588-6755190bf05b service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:25.266638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:25.269767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:25.270392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:25.271022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] PATH_INFO: `/v3/users/414eb727b79b403380752cc4c8eaa575` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:25.273355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=414eb727b79b403380752cc4c8eaa575)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:25.277583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:25.388727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414eb727b79b403380752cc4c8eaa575'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.390482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414eb727b79b403380752cc4c8eaa575'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.402002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414eb727b79b403380752cc4c8eaa575'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.416786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414eb727b79b403380752cc4c8eaa575'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.437199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-510f511c-274f-463e-95da-15fd86febe8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414eb727b79b403380752cc4c8eaa575'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.449508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1009/2012] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:25 2022] DELETE /identity/v3/users/414eb727b79b403380752cc4c8eaa575 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:25.503635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:25.505883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:25.522826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe502ce-2688-44c0-bdb5-2d376a389291 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:25.685487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:25.686617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:25.686895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:25.687163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] PATH_INFO: `/v3/projects/6c2e1cfab8324f15880a1ad2f5adea3d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:25.690375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2e1cfab8324f15880a1ad2f5adea3d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:25.692467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:25.832497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2e1cfab8324f15880a1ad2f5adea3d', 'request_id': 'req-f65d94a5-27dd-4aa1-952c-76b9907a91ec'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.835610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:25.847879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2e1cfab8324f15880a1ad2f5adea3d', 'request_id': 'req-f65d94a5-27dd-4aa1-952c-76b9907a91ec'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.862051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2e1cfab8324f15880a1ad2f5adea3d', 'request_id': 'req-f65d94a5-27dd-4aa1-952c-76b9907a91ec'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.881810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] The token cache is being invalidate because project 6c2e1cfab8324f15880a1ad2f5adea3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:25.882094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.882922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:25.887976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1004/2013] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:25 2022] DELETE /identity/v3/projects/6c2e1cfab8324f15880a1ad2f5adea3d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:25.895656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acd6-ee3f-4ab8-986b-8e478c786090 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:25.896190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acd6-ee3f-4ab8-986b-8e478c786090 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:25.896663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11acd6-ee3f-4ab8-986b-8e478c786090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:25.938290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0d11acd6-ee3f-4ab8-986b-8e478c786090 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:25.939582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d11acd6-ee3f-4ab8-986b-8e478c786090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:25.975820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:25.980453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.076821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:26.084268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d94a5-27dd-4aa1-952c-76b9907a91ec admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.097828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.098651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.098940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.105328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] PATH_INFO: `/v3/projects/bc349397d8fe4d829493fc44d4fd478d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.106847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc349397d8fe4d829493fc44d4fd478d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.110039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.122340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1010/2014] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:26.126567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.126969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.127181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.162251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:26.163933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.197763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc349397d8fe4d829493fc44d4fd478d', 'request_id': 'req-c207bd79-0fd8-44c5-8584-bccac4ee686c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:26.216209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc349397d8fe4d829493fc44d4fd478d', 'request_id': 'req-c207bd79-0fd8-44c5-8584-bccac4ee686c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:26.237035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc349397d8fe4d829493fc44d4fd478d', 'request_id': 'req-c207bd79-0fd8-44c5-8584-bccac4ee686c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:26.265455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] The token cache is being invalidate because project bc349397d8fe4d829493fc44d4fd478d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:26.266133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:26.267557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-c207bd79-0fd8-44c5-8584-bccac4ee686c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:26.277684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1005/2015] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:25 2022] DELETE /identity/v3/projects/bc349397d8fe4d829493fc44d4fd478d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:26.277916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1011/2016] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:26.278367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2ff8-6a10-4af2-93e0-4e8e9cb23632 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.278367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2ff8-6a10-4af2-93e0-4e8e9cb23632 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.278367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2ff8-6a10-4af2-93e0-4e8e9cb23632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.280579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.294634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dfef6-806f-4d11-8226-fc18ad268e0a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.300641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.301288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.301564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.301833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.302516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:26.302818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.304922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.316906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c72f2ff8-6a10-4af2-93e0-4e8e9cb23632 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:26.320213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c72f2ff8-6a10-4af2-93e0-4e8e9cb23632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.326250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1012/2017] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:54:26.332337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.334770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.340451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1006/2018] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:26.349713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.349987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.351859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.354077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d95664-c793-4de2-8c2e-a5445d109631 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.359432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.360469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.360863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.361138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.361950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.365746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.384554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:26.386172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.405974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1007/2019] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:26.407285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.409172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1013/2020] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:26.413916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.422130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d4285-2f7b-48d2-8f8e-954478873f90 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.509471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:26.519345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.520026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.520510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.521522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.522281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:26.522810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.524756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.524911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d39b1-480e-4707-a106-5df0b94c8b24 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.530366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.531043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.531365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.531642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.532290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.534344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.539805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1008/2021] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:26.548570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.561601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1014/2022] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:26.565962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.572692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cc1a5-8952-43aa-b82e-1c01c9dabc7e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.577229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a3a2e-f2c0-4711-98c4-027b80ef6a2c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.581551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.582813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.582952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.583387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.583513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.583749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.583912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.584008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.584674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.586335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.586487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.591174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.601229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1015/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:26.606767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.625367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760d6d8-6b9c-4a2c-9247-f17163a25a9f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.632136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.632785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.633067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.633343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.634010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.636089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.650512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1016/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:26.656453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.674855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb4dd-8e70-4b6c-ac1d-aa8a6de354c1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.684334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1009/2025] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:26.685072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:26.685903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.686040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.686310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] PATH_INFO: `/v3/projects/880f7d1c88224ebd85d15f7c43e7fe72/users/e76b3353ce4b4361872ed6dba9f895d6/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.708612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880f7d1c88224ebd85d15f7c43e7fe72, user_id=e76b3353ce4b4361872ed6dba9f895d6, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:26.711827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97718a3f-4a9f-43ae-aab1-51f72cbfcfb0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:26.719771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.741298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1017/2026] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:26 2022] PUT /identity/v3/projects/880f7d1c88224ebd85d15f7c43e7fe72/users/e76b3353ce4b4361872ed6dba9f895d6/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:26.744692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:26.745064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:26.745426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:26.769173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:26.771800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:26.790014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:26.912602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586807-22ac-428b-bead-86bbe097b976 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:26.934652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1018/2027] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:26.950166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:26.953994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:27.113572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:27.118016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.118395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.118631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.121507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:27.126722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:27.167367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be989d-4bf9-4086-a6f8-49ac61c3c2ca None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:27.171915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:27.172614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.172891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.173199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.183700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1010/2028] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:26 2022] POST /identity/v3/users => generated 287 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:27.189373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:27.191018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:27.192409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:27.207557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1019/2029] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:27.228582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60cce5-f26b-4da8-95ab-14883cd16cdb admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:27.259283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:27.260915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.261695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.262548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.266194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:27.271546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:27.353801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1011/2030] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:27.357665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:27.391690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c837-6aa8-4bbb-8cce-94a27fe0d3aa service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:27.399205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:27.400318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.400318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.400565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.401235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:27.407669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a461-a550-4fbf-acdf-ad4d7795a407 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:27.446485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1020/2031] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:27.450584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:27.479740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833bdc-a287-4def-9f63-8faa7d114c64 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:27.491622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:27.492326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.495339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.495648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] PATH_INFO: `/v3/projects/baab1921435a4554b2de19f97c41df5f/users/20d79e4c3e72404f85cbe78dbb1e3fde/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.534800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baab1921435a4554b2de19f97c41df5f, user_id=20d79e4c3e72404f85cbe78dbb1e3fde, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:27.537678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:27.575961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1012/2032] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:27 2022] PUT /identity/v3/projects/baab1921435a4554b2de19f97c41df5f/users/20d79e4c3e72404f85cbe78dbb1e3fde/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:27.581364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330dbbd-0310-4511-a530-d38513ff909d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:27.581728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330dbbd-0310-4511-a530-d38513ff909d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:27.581995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330dbbd-0310-4511-a530-d38513ff909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:27.609769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0330dbbd-0310-4511-a530-d38513ff909d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:27.616931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0330dbbd-0310-4511-a530-d38513ff909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:27.744847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1021/2033] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:27.779742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:27.781559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:27.943859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:28.037256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28896b72-91dc-4219-a284-2b559101c898 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:28.048652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:28.049791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:28.051267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:28.051807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:28.082100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:28.084391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:28.102154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1013/2034] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:29.870543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:29.871768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:29.872369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:29.933369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] MFA Rules not processed for user `20d79e4c3e72404f85cbe78dbb1e3fde`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:29.938023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:30.047517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:30.187820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1022/2035] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:30.188242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.201677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc676e28-7c4b-48c2-a99c-bde06307a209 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.241340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.241991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.242273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.242990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.254984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.279976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.361756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.364062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1014/2036] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:30 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:30.385208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12eecd-543a-4a6c-b1b6-8beaf3f3b8b4 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.502372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.503513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.503723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.504042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.504911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.508997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.547903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1023/2037] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:30 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:30.552118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.572901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719300ec-8720-4dc1-93e1-1b7fbaa3da21 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.586767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.588595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.588999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.589505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.590938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.594488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.618911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1015/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:30.623421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.637916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6621f4-bb96-44c4-81bf-3fcd7d75090d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.648922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.650259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.650890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.651531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] PATH_INFO: `/v3/projects/1a41068bd73247eeaf090fc1878a46a8/users/a6ab20d710d54250be2fd29f81f4dd2d/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.721055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a41068bd73247eeaf090fc1878a46a8, user_id=a6ab20d710d54250be2fd29f81f4dd2d, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.731246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.741296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.744164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:30.757313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1024/2039] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:30 2022] PUT /identity/v3/projects/1a41068bd73247eeaf090fc1878a46a8/users/a6ab20d710d54250be2fd29f81f4dd2d/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:30.764363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:30.777257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47a12-4b27-41ce-a38d-c97dc0966978 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.942750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.943885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.944019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f29d21-29c8-4226-81de-3b92c4213cb4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:30.944779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.945338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] PATH_INFO: `/v3/users/cac271e4b1194566a0ac2d1e669fe9ed` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.947392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac271e4b1194566a0ac2d1e669fe9ed)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.949933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.960569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:30.961218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:30.961635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:30.961916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:30.962596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:30.964360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:30.981177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1025/2040] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:30.982355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:31.005845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22789-1849-43a3-8674-17eaa2da9c04 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:31.014453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:31.015416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:31.015700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:31.016254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] PATH_INFO: `/v3/projects/1a41068bd73247eeaf090fc1878a46a8/users/a6ab20d710d54250be2fd29f81f4dd2d/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:31.026063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a41068bd73247eeaf090fc1878a46a8, user_id=a6ab20d710d54250be2fd29f81f4dd2d, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:31.032234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69b9db-bf21-4f86-810c-0a3e02d3633e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:31.052816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac271e4b1194566a0ac2d1e669fe9ed'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:31.054800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac271e4b1194566a0ac2d1e669fe9ed'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:31.056801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac271e4b1194566a0ac2d1e669fe9ed'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:31.064265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1026/2041] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:30 2022] PUT /identity/v3/projects/1a41068bd73247eeaf090fc1878a46a8/users/a6ab20d710d54250be2fd29f81f4dd2d/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:31.096786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac271e4b1194566a0ac2d1e669fe9ed'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:31.131371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac271e4b1194566a0ac2d1e669fe9ed'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:31.158788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1016/2042] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:30 2022] DELETE /identity/v3/users/cac271e4b1194566a0ac2d1e669fe9ed => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:31.164336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:31.164636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:31.164908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:31.202342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:31.203724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:31.229824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:31.325447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1027/2043] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:31.463361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:31.465580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:31.579865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:31.666082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821f93a-d58a-4da8-8b06-5b55ca6a266b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:31.681408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:31.682612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:31.686037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:31.686898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:31.704674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:31.707284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:31.724882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1017/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:32.898420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:32.900734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:32.935172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d61c3-b17d-4a3e-b1f5-4e97834f442a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:32.948311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:32.952180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:32.952753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:32.953213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] PATH_INFO: `/v3/projects/de4696abe6884fc8af9d48e58a19be7a` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:32.956745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de4696abe6884fc8af9d48e58a19be7a)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:32.961703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:33.070760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de4696abe6884fc8af9d48e58a19be7a', 'request_id': 'req-0370ff76-5d89-47bb-b525-bc79af663e6e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.075474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de4696abe6884fc8af9d48e58a19be7a', 'request_id': 'req-0370ff76-5d89-47bb-b525-bc79af663e6e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.084135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de4696abe6884fc8af9d48e58a19be7a', 'request_id': 'req-0370ff76-5d89-47bb-b525-bc79af663e6e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.108301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] The token cache is being invalidate because project de4696abe6884fc8af9d48e58a19be7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:33.108948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.109950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-0370ff76-5d89-47bb-b525-bc79af663e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.116150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1028/2045] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:32 2022] DELETE /identity/v3/projects/de4696abe6884fc8af9d48e58a19be7a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:33.121078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:33.139510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209774-d36c-49f1-9f11-025aa57222d3 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:33.286369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:33.287689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:33.288230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:33.288782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] PATH_INFO: `/v3/users/9a9f7e31ee6f4295aec0c43d141028d7` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:33.291202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a9f7e31ee6f4295aec0c43d141028d7)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:33.294800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:33.393619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:33.397142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:33.397665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:33.421424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a9f7e31ee6f4295aec0c43d141028d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.422569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a9f7e31ee6f4295aec0c43d141028d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.423541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a9f7e31ee6f4295aec0c43d141028d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.442236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a9f7e31ee6f4295aec0c43d141028d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.476950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a9f7e31ee6f4295aec0c43d141028d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:33.503362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1018/2046] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:33 2022] DELETE /identity/v3/users/9a9f7e31ee6f4295aec0c43d141028d7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:33.516414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] MFA Rules not processed for user `a6ab20d710d54250be2fd29f81f4dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:33.526015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:33.801410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:33.802430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1029/2047] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:33.803602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:33.946511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:33.996812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40132-15ce-4a98-ae45-971ddbbb4d72 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:34.020722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:34.022481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:34.023059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:34.026854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:34.039541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:34.256799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:34.263123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff275c-918f-43bb-98cd-02de77658f64 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:34.302437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1019/2048] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:34.982556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:34.985922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:35.153565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:35.251826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45f782-e930-45cf-b42a-46c67c37aa8c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:35.260607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:35.263352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.263803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.264259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] PATH_INFO: `/v3/projects/0babeac690fa4262b997bbe5520a5828` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.275549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0babeac690fa4262b997bbe5520a5828)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:35.304280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:35.373096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.373919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.373919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.440700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0babeac690fa4262b997bbe5520a5828', 'request_id': 'req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:35.443081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] MFA Rules not processed for user `e76b3353ce4b4361872ed6dba9f895d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:35.444607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:35.461383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0babeac690fa4262b997bbe5520a5828', 'request_id': 'req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:35.485424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0babeac690fa4262b997bbe5520a5828', 'request_id': 'req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:35.540265 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] The token cache is being invalidate because project 0babeac690fa4262b997bbe5520a5828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:35.540737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:35.542138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ab561ac6-1373-47ea-9b39-fd7d7cb9330c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:35.551317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1030/2049] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:34 2022] DELETE /identity/v3/projects/0babeac690fa4262b997bbe5520a5828 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:35.557239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba7d2-c99f-4290-a23f-461e815166c6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.557718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba7d2-c99f-4290-a23f-461e815166c6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.558667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ba7d2-c99f-4290-a23f-461e815166c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.601532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-187ba7d2-c99f-4290-a23f-461e815166c6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:35.604131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-187ba7d2-c99f-4290-a23f-461e815166c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:35.610580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1020/2050] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:35.611838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:35.613422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:35.774609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1031/2051] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:35.783360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139514b-ef80-4ee7-bb16-3754230e987b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:35.788865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.789139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.789539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.793262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:35.794411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.794898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.795475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.797838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:35.802508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:35.814044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:35.815624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:35.834514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1032/2052] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:35.835685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:35.847050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095a50-d3b8-45fc-be5d-4cf54e3a5667 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:35.852668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:35.853362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:35.853644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:35.853951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:35.854945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:35.855440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:35.857190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:35.873851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1021/2053] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:35.875812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1033/2054] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:35.883413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:35.883640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:35.905026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef462-0d93-4e81-b361-783ca7ba05ad admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:35.916602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:35.978391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:36.007791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.008510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.008905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.009021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.010241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.012605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.082129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1034/2055] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:36.082988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c56339-fb4a-4145-8595-f480aef7db49 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.093596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.250103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553be-efe2-44b4-b6e7-a2960ada510a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.263969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.264720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.265062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.265388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.266123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.268240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.274184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.275240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.275763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.276230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.277615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.284588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.310306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1035/2056] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:36 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:36.310306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.310306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:36.332984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1022/2057] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:35 2022] POST /identity/v3/users => generated 299 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:36.332984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.363039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e41d4-1417-4546-8c2e-7a498d27ef24 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.375443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.380149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.381219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.382064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.385755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.390703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.416545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1023/2058] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:36.417767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.435183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034391f1-c20c-4a5d-ae4d-3528aba45334 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.444246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.445515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.446094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.446655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.447999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.451170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.452221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e9df7-6d4e-461b-932a-6bc1e741c2d6 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.461551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.462413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.463151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.463678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.466180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:36.472511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1024/2059] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:36.473889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.492078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2190-7f9c-4a95-9845-eda1d5ca0ecb admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.501335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.502592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.503206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.503605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.504801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.507874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.528213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1025/2060] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:36.529802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.547829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818ecf-9422-4023-a61b-31a995cf5752 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.549010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.550565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.556437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.557928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.558613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.559091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] PATH_INFO: `/v3/projects/b1da04c606664831adb0d1b12d7815b2/users/3ec6216ba1c64fdca97e3705b683f999/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.561431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1036/2061] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:36.562372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.570511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:36.575495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53376f-8cfb-419f-a8b7-942d97672c1d service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.585130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.586264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.587292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.587292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] PATH_INFO: `/v3/projects/21857b99def94fd59692266951e0f6db/users/7e52fc83b068443587a6ef1ed5c2196b/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.592455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1da04c606664831adb0d1b12d7815b2, user_id=3ec6216ba1c64fdca97e3705b683f999, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.596427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.617068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=21857b99def94fd59692266951e0f6db, user_id=7e52fc83b068443587a6ef1ed5c2196b, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.624561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb41a-8450-4920-818b-cd39e400bbaf admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.626325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1026/2062] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:36 2022] PUT /identity/v3/projects/b1da04c606664831adb0d1b12d7815b2/users/3ec6216ba1c64fdca97e3705b683f999/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:36.631286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.657561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1037/2063] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:36 2022] PUT /identity/v3/projects/21857b99def94fd59692266951e0f6db/users/7e52fc83b068443587a6ef1ed5c2196b/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:36.669654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.670125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.672285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.748992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:36.773683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673c45-953d-4eb0-9d43-5826bc7ab9e3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:36.780236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] MFA Rules not processed for user `7e52fc83b068443587a6ef1ed5c2196b`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:36.792332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:36.929518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:36.930487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:36.930788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:36.931179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:36.934605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:36.936948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:36.956878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1027/2064] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:36.959983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:36.993137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923631b-c651-48e0-ab21-fbfbaf7e9e0a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.034129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.035801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.036660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.038292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] PATH_INFO: `/v3/projects/b1da04c606664831adb0d1b12d7815b2/users/3ec6216ba1c64fdca97e3705b683f999/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.051932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1da04c606664831adb0d1b12d7815b2, user_id=3ec6216ba1c64fdca97e3705b683f999, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.058571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.101136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1028/2065] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:36 2022] PUT /identity/v3/projects/b1da04c606664831adb0d1b12d7815b2/users/3ec6216ba1c64fdca97e3705b683f999/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:37.170801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1038/2066] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:37.178123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.309979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c02bfe-d7c2-4b1e-a260-e270cdc8639b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.325231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.326356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.326873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.327525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.345025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.353542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.439169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1029/2067] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:37 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:37.440505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.454546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929b30-4bf0-4a24-b03f-ede69f39ef3d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.602146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.603616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.603968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.604293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.605137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.607804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.641820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1039/2068] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:37 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:37.646869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.667160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e587d2-140e-4378-885a-dfe83ad51b3f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.675491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.676887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.677448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.678140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.679675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.681812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.704793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1030/2069] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:37.706778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.725716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c5b26-d9d7-4b41-97e8-ebe313f83053 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.733781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.735029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.735627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.736124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] PATH_INFO: `/v3/projects/4c768adff8f64746be795a4d52a63405/users/c503786c3d754a8ba4b448d29f71aabe/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.766403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c768adff8f64746be795a4d52a63405, user_id=c503786c3d754a8ba4b448d29f71aabe, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.772188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.805245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1040/2070] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:37 2022] PUT /identity/v3/projects/4c768adff8f64746be795a4d52a63405/users/c503786c3d754a8ba4b448d29f71aabe/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:37.807740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.829115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a463e-831a-41cf-8914-90da46c36bfd admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.928136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:37.936741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.937404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.937758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.938124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:37.938705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:37.941760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa3862-c424-4160-9b8c-8145a4e74a91 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:37.957784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1031/2071] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:37.964587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:37.987730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0e806-9284-451b-ae55-8accdffbae83 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:37.996394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:37.998315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:37.998860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:37.999450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] PATH_INFO: `/v3/projects/4c768adff8f64746be795a4d52a63405/users/c503786c3d754a8ba4b448d29f71aabe/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:38.003033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c768adff8f64746be795a4d52a63405, user_id=c503786c3d754a8ba4b448d29f71aabe, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:38.007730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:38.024451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1041/2072] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:37 2022] PUT /identity/v3/projects/4c768adff8f64746be795a4d52a63405/users/c503786c3d754a8ba4b448d29f71aabe/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:38.037453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:38.037975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:38.039147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:38.114720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] MFA Rules not processed for user `c503786c3d754a8ba4b448d29f71aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:38.117264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:38.264437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1032/2073] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:38.279838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:38.332624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:38.429838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb89efe-e9a4-4aee-9fef-2af909cc81f8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:38.453037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:38.453685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:38.454111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:38.454250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:38.479598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:38.510701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:38.556271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1042/2074] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:39.572068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:39.588826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0976ca-5b12-4ce4-be92-d55ebf7f6cf6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:39.598647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c503786c3d754a8ba4b448d29f71aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c768adff8f64746be795a4d52a63405', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:39.599744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:39.600245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:39.600704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:39.601964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:39.602914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:39.605589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:39.637173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1033/2075] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:39.638689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:39.657945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf688c-c04f-4a84-aa20-196a289e91d9 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:39.666321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c503786c3d754a8ba4b448d29f71aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c768adff8f64746be795a4d52a63405', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:39.668483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:39.668815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:39.670369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:39.671303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:39.674288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:39.714593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1043/2076] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:39 2022] POST /identity/v3/projects => generated 355 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:39.720228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:39.742649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3263616-1184-4330-ac8b-33f4af515306 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:39.964808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c503786c3d754a8ba4b448d29f71aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c768adff8f64746be795a4d52a63405', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:39.981852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:39.982273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:39.982774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:39.984023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:39.984546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:39.989424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:40.017466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1034/2077] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:40.076333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:40.090407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3b3e4-b92e-4294-88ca-25ffba6b2e01 tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:40.098083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c503786c3d754a8ba4b448d29f71aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c768adff8f64746be795a4d52a63405', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:40.098758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:40.099088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:40.099456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] PATH_INFO: `/v3/projects/4097d2f6f5db4b3b8cadc3b370b85735` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:40.101099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4097d2f6f5db4b3b8cadc3b370b85735)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:40.104132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:40.192835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4097d2f6f5db4b3b8cadc3b370b85735', 'request_id': 'req-71d499c1-349d-45a2-a6c5-8565f51bb91d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:40.204416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4097d2f6f5db4b3b8cadc3b370b85735', 'request_id': 'req-71d499c1-349d-45a2-a6c5-8565f51bb91d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:40.213478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4097d2f6f5db4b3b8cadc3b370b85735', 'request_id': 'req-71d499c1-349d-45a2-a6c5-8565f51bb91d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:40.249382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] The token cache is being invalidate because project 4097d2f6f5db4b3b8cadc3b370b85735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:40.250642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:40.253378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-71d499c1-349d-45a2-a6c5-8565f51bb91d tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:40.256296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1044/2078] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:40 2022] DELETE /identity/v3/projects/4097d2f6f5db4b3b8cadc3b370b85735 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:40.693296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:40.698251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:40.804837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:40.819067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4f51d-ab43-4dce-96ab-06bfa6c969ae tempest-VolumeQuotaClassesTest-1095562819 tempest-VolumeQuotaClassesTest-1095562819-project-admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:40.927436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:40.930007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:40.930845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:40.931392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] PATH_INFO: `/v3/users/7e52fc83b068443587a6ef1ed5c2196b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:40.933121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e52fc83b068443587a6ef1ed5c2196b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:40.935461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:41.041380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e52fc83b068443587a6ef1ed5c2196b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:41.042309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e52fc83b068443587a6ef1ed5c2196b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:41.045600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e52fc83b068443587a6ef1ed5c2196b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:41.081358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e52fc83b068443587a6ef1ed5c2196b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:41.109149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e52fc83b068443587a6ef1ed5c2196b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:41.135979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1035/2079] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:40 2022] DELETE /identity/v3/users/7e52fc83b068443587a6ef1ed5c2196b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:41.139519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:41.139519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:41.139519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:41.190243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:41.196771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:41.466514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1045/2080] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:41.620008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:41.622064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:41.820993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6015-48e3-4705-bf88-612a11e0422b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:41.850937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:41.852021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:41.852490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:41.853144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:41.904584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:41.906094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa197-998d-47f6-b85d-6f524b00d985 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:41.938317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1036/2081] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:44.502404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:44.505663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:44.531753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657289c9-045b-4296-8a86-df7b3d7e0359 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:44.545081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:44.547053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:44.547389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:44.548392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] PATH_INFO: `/v3/projects/21857b99def94fd59692266951e0f6db` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:44.550243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21857b99def94fd59692266951e0f6db)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:44.552853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:44.622737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:44.702065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21857b99def94fd59692266951e0f6db', 'request_id': 'req-4b843597-c53a-4bc1-8e20-c68372b69bc8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:44.712978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21857b99def94fd59692266951e0f6db', 'request_id': 'req-4b843597-c53a-4bc1-8e20-c68372b69bc8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:44.721172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:44.727801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21857b99def94fd59692266951e0f6db', 'request_id': 'req-4b843597-c53a-4bc1-8e20-c68372b69bc8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:44.728321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:44.729846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:44.757407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] The token cache is being invalidate because project 21857b99def94fd59692266951e0f6db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:44.757735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:44.759709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:44.761938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1046/2082] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:44 2022] DELETE /identity/v3/projects/21857b99def94fd59692266951e0f6db => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:44.766489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:44.770737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:44.836085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] MFA Rules not processed for user `3ec6216ba1c64fdca97e3705b683f999`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:44.841104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:44.878096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:44.978457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843597-c53a-4bc1-8e20-c68372b69bc8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:45.010799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:45.011764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:45.012726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:45.013099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] PATH_INFO: `/v3/users/c503786c3d754a8ba4b448d29f71aabe` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:45.014735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c503786c3d754a8ba4b448d29f71aabe)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:45.017261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:45.104855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1037/2083] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:45.149210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:45.154585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:45.179577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c503786c3d754a8ba4b448d29f71aabe'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:45.182860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c503786c3d754a8ba4b448d29f71aabe'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:45.207809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c503786c3d754a8ba4b448d29f71aabe'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:45.224750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c503786c3d754a8ba4b448d29f71aabe'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:45.256056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c503786c3d754a8ba4b448d29f71aabe'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:45.279273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1047/2084] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:44 2022] DELETE /identity/v3/users/c503786c3d754a8ba4b448d29f71aabe => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:45.291251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5b749-d912-4420-a68e-f0abaee467d6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:45.443772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:45.444443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:45.444717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:45.444991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:45.450122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:45.685389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:45.689087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67862063-57a4-48ad-9ff7-c1533bef26cc service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:45.720503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1038/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:46.127197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:46.129761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:46.215399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:46.308821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639be3-d6a9-4693-8fb7-791ca43bb984 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:46.320706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:46.321410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.321692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.322051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] PATH_INFO: `/v3/projects/4c768adff8f64746be795a4d52a63405` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.325687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c768adff8f64746be795a4d52a63405)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:46.327681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:46.328962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.331489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.332279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.376772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:46.379205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:46.412147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:46.414365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c768adff8f64746be795a4d52a63405', 'request_id': 'req-41ebf013-bf64-4e45-94a1-f545a94e319b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:46.425283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c768adff8f64746be795a4d52a63405', 'request_id': 'req-41ebf013-bf64-4e45-94a1-f545a94e319b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:46.433861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c768adff8f64746be795a4d52a63405', 'request_id': 'req-41ebf013-bf64-4e45-94a1-f545a94e319b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:46.468674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] The token cache is being invalidate because project 4c768adff8f64746be795a4d52a63405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:46.469079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:46.469919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-41ebf013-bf64-4e45-94a1-f545a94e319b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:46.472754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1048/2086] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:46 2022] DELETE /identity/v3/projects/4c768adff8f64746be795a4d52a63405 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:46.481104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838addf-b05e-4862-ba05-99be19eaf839 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.483462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838addf-b05e-4862-ba05-99be19eaf839 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.483952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838addf-b05e-4862-ba05-99be19eaf839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.516321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0838addf-b05e-4862-ba05-99be19eaf839 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:46.517635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0838addf-b05e-4862-ba05-99be19eaf839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:46.540558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1039/2087] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:54:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:46.616453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:46.620481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:46.622927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1049/2088] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:46.633370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.633809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.634322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.674253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:46.678367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:46.706581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1050/2089] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:46.720490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:46.757732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fec820-0ae1-48ec-9a59-8b6d6e1a4589 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:46.766935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:46.767768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.769695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.770129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.774355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:46.774826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:46.777104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:46.815185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1051/2090] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:46.816459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:46.848596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c12cf-7faf-4ae5-8c12-a3102ad3571c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:46.862336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:46.862978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:46.863281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:46.863562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:46.866917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:46.870749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:46.889367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15200de-e3dc-4386-a13f-9e4c2675b2c6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:46.961304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1052/2091] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:46 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:46.966428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:47.032683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:47.094243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:47.094989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.095300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.095625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:47.275785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f86cb0-f8eb-44c7-ac9a-b59d07295d5e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:47.288235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:47.288768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:47.289286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.289728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.290159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:47.291292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:47.295965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:47.296621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:47.332482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1040/2092] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:54:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:54:47.366259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1053/2093] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:46 2022] POST /identity/v3/users => generated 293 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:47.366988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:47.402554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7368-d39e-41a2-aac4-9185d3e876f4 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:47.430901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:47.433138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.437122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.437458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:47.443270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:47.445868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:47.489599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:47.527569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1041/2094] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:54:47.538724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:47.575662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e6db-d230-4745-8c09-fb963ee0bb23 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:47.584972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:47.586344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.586847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.587375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:47.589263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:47.592419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef1ea4-243e-415c-9b69-1a2bd834d97a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:47.625163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1054/2095] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:47.630940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:47.668483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1448006-7790-4e8b-8fdc-221c143fec21 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:47.680100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:47.691470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.692050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.692521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] PATH_INFO: `/v3/projects/f7f4f3447942486b9e5019a43b4e5659/users/8c444d45b8194f9ca4c46a55570a7717/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:47.738601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f4f3447942486b9e5019a43b4e5659, user_id=8c444d45b8194f9ca4c46a55570a7717, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:47.747438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:47.825717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1042/2096] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:47 2022] PUT /identity/v3/projects/f7f4f3447942486b9e5019a43b4e5659/users/8c444d45b8194f9ca4c46a55570a7717/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:47.845427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:47.846081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:47.846663 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:48.015335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] MFA Rules not processed for user `8c444d45b8194f9ca4c46a55570a7717`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:48.017949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:48.259996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:48.261517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1055/2097] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:48.522319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8792-7cc3-4dd0-84d3-37ea889bad8e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:48.540863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:48.542081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:48.542677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:48.543721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:48.544650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:48.548850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:48.641130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1043/2098] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:48 2022] POST /identity/v3/projects => generated 387 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:48.647504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:48.674675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854ca87-3f38-452e-bb5d-e0b6e9b371b5 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:48.953806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:48.954474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:48.954760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:48.955678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:48.956419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:48.960732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:49.044253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1056/2099] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:48 2022] POST /identity/v3/users => generated 294 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:49.048648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:49.083608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b373de-857a-477f-a81d-bb21d953b14f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:49.104801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:49.108038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:49.110888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:49.111461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:49.113222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:49.122796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:49.169386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1044/2100] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:49.174262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:49.214637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d3a8-79ee-4fc4-af0a-86a75f8ad6f3 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:49.241925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:49.245884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:49.246158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:49.247649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:49.253576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:49.255182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135ee7-50f9-43ef-ac11-488b773c8046 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:49.287892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1057/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:49.292741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:49.307825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c51108-33f3-4dc5-8787-b83d68ed47c9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:49.315005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:49.315773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:49.316164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:49.316520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] PATH_INFO: `/v3/projects/57ef4ad6675d4e7e97d120f17e0be345/users/4d1e49f684474b8b9d3fdb89915d34a8/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:49.346717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ef4ad6675d4e7e97d120f17e0be345, user_id=4d1e49f684474b8b9d3fdb89915d34a8, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:49.349708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:49.390322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1045/2102] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:49 2022] PUT /identity/v3/projects/57ef4ad6675d4e7e97d120f17e0be345/users/4d1e49f684474b8b9d3fdb89915d34a8/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:49.417680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:49.418138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:49.418559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:49.444025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:49.557571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] MFA Rules not processed for user `4d1e49f684474b8b9d3fdb89915d34a8`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:49.560042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:49.791856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1058/2103] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:49.807305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:50.384247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d798f-b7eb-4078-bf4b-500f248358d4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:50.417529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:50.423170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:50.424033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:50.424583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:50.430620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:50.443406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:50.582311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:50.590585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1046/2104] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:49 2022] POST /identity/v3/projects => generated 387 bytes in 790 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:50.616120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f78ba2-0e8f-405e-a02e-5b332b66cc9f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:50.853358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:50.856185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:50.857629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:50.857943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:50.859850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:50.868328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:50.958374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:50.960427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1059/2105] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:50 2022] POST /identity/v3/users => generated 300 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:51.006737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581fed7-435d-4c72-aacc-ffe8da578627 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:51.016142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:51.033004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:51.033615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:51.034190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:51.048690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:51.065768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:51.114128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:51.139681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:51.148784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1047/2106] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:51.173804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884daa-b5c9-4077-b9b4-5a77ca1a85af admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:51.201476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:51.203574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:51.204119 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:51.204654 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] PATH_INFO: `/v3/projects/cb981d74a4c04fc890148ed7ed4b962c/users/8bcf888b7cd4493594dcf5ec4364f150/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:51.292914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb981d74a4c04fc890148ed7ed4b962c, user_id=8bcf888b7cd4493594dcf5ec4364f150, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:51.298457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:51.342893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1060/2107] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:51 2022] PUT /identity/v3/projects/cb981d74a4c04fc890148ed7ed4b962c/users/8bcf888b7cd4493594dcf5ec4364f150/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:51.350135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:51.387991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f653e-0562-4ab5-a402-8c8d1209fb45 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:51.578621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:51.579992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:51.579992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:51.579992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:51.580900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:51.582652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093aea5-8940-4361-a498-f7fabe2d4912 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:51.607193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1048/2108] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:51.612819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:51.654831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2633b4b-b0b2-44cb-9174-f9880e5b9f49 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:51.679162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:51.680974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:51.681675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:51.682468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] PATH_INFO: `/v3/projects/cb981d74a4c04fc890148ed7ed4b962c/users/8bcf888b7cd4493594dcf5ec4364f150/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:51.701614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb981d74a4c04fc890148ed7ed4b962c, user_id=8bcf888b7cd4493594dcf5ec4364f150, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:51.709446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:51.738386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1061/2109] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:51 2022] PUT /identity/v3/projects/cb981d74a4c04fc890148ed7ed4b962c/users/8bcf888b7cd4493594dcf5ec4364f150/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:51.764563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adadf66-e4e9-4f17-b5c7-ffa8e32e08a3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:51.764862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adadf66-e4e9-4f17-b5c7-ffa8e32e08a3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:51.765150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adadf66-e4e9-4f17-b5c7-ffa8e32e08a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:51.864337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6adadf66-e4e9-4f17-b5c7-ffa8e32e08a3 None None] MFA Rules not processed for user `8bcf888b7cd4493594dcf5ec4364f150`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:51.868507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6adadf66-e4e9-4f17-b5c7-ffa8e32e08a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:52.211817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1049/2110] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:52.360738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:52.364104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:52.613970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:52.631136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656e9f-3a18-46f2-9ef5-72c178b74986 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:52.640097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264bdb3-566b-4724-88dc-170e36b2b75e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:52.640097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264bdb3-566b-4724-88dc-170e36b2b75e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:52.641368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264bdb3-566b-4724-88dc-170e36b2b75e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:52.643496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1050/2111] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:54:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:54:52.649369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:52.650433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:52.650852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:52.663254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:52.663578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf788-4564-4efb-a41b-95515172a4a1 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:52.663578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf788-4564-4efb-a41b-95515172a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:52.663578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf788-4564-4efb-a41b-95515172a4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:52.715935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:52.722755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:52.749141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-1badf788-4564-4efb-a41b-95515172a4a1 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:52.751118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1badf788-4564-4efb-a41b-95515172a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:52.786674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1062/2112] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:52.871045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:52.874857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:53.044964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1051/2113] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:54:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:53.066700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483800d4-90df-4c80-aecd-5e5b045bd3d1 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:53.067279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483800d4-90df-4c80-aecd-5e5b045bd3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:53.067804 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483800d4-90df-4c80-aecd-5e5b045bd3d1 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:53.069887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1052/2114] 10.222.0.26 () {54 vars in 961 bytes} [Mon Sep 12 23:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:54:53.102480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:53.103992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:53.104657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:53.164334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541a44-5df6-4480-bb4e-f6998d2b499e service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:53.177582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:53.178578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:53.180474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:53.181353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] PATH_INFO: `/v3/users/d46cf7ae501740869d677d0e9e007041` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:53.186709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46cf7ae501740869d677d0e9e007041)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:53.190992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:53.201661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:53.212029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:53.258379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1053/2115] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Sep 12 23:54:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:54:53.287086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:53.330865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c2864-4305-4dfc-9b89-7b283db3d26f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:53.376854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:53.379028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:53.380313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:53.383875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:53.392802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46cf7ae501740869d677d0e9e007041'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:53.395873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46cf7ae501740869d677d0e9e007041'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:53.418034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:53.420086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:53.435197 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46cf7ae501740869d677d0e9e007041'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:53.437305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:53.459245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1054/2116] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:54:53.463269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46cf7ae501740869d677d0e9e007041'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:53.509300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46cf7ae501740869d677d0e9e007041'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:53.525932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1063/2117] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:52 2022] DELETE /identity/v3/users/d46cf7ae501740869d677d0e9e007041 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:54.059480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:54.122356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d71a66-22ed-4708-a608-97bea6069ff9 service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:54.362995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:54.368192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:54.370880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:54.371520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] PATH_INFO: `/v3/projects/58a7b6a5dc86486682777285fa3295ac` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:54.388751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a7b6a5dc86486682777285fa3295ac)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:54.402204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:54.571473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:54.574405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:54.576928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a7b6a5dc86486682777285fa3295ac', 'request_id': 'req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:54.588956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a7b6a5dc86486682777285fa3295ac', 'request_id': 'req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:54.601686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a7b6a5dc86486682777285fa3295ac', 'request_id': 'req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:54.622591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] The token cache is being invalidate because project 58a7b6a5dc86486682777285fa3295ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:54.623043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:54.624590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a1bea2d8-2ae9-4849-ac98-4de8b2640e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:54.632309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1055/2118] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:54 2022] DELETE /identity/v3/projects/58a7b6a5dc86486682777285fa3295ac => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:54.636553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55812-dd61-4b74-a7fc-5d488bd232be None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:54.637234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55812-dd61-4b74-a7fc-5d488bd232be None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:54.637658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c55812-dd61-4b74-a7fc-5d488bd232be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:54.672224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-90c55812-dd61-4b74-a7fc-5d488bd232be None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:54.675926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-90c55812-dd61-4b74-a7fc-5d488bd232be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:54.817420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c6674-7954-4186-a755-30e88dfe1240 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:54.819642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1056/2119] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:54.822388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:54.824969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:54.826060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:54.868188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:54.870338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:54.875750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:54.881752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1057/2120] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:54.890354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:54.913628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bb53a-4f76-4f40-9006-da0042efdc78 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:54.937706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:54.938493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:54.938776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:54.939060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:54.939772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:54:54.940146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:54.942983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:54.964347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1058/2121] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:54:54.966363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:54.990704 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075de01-707c-4beb-acc4-19652dcb3705 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:54.993069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:54.993464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:54.993919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:54.994374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:54.998201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:55.002380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:55.003159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.003468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.003740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.004400 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.009793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.029196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:55.125315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1059/2122] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:54:54 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:54:55.127840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:55.219571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.222646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.252717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025ed23-c664-4498-8ffc-07414c4852b5 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:55.264965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:55.265811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.266217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.266701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.267528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.270622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.274425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1064/2123] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:55.325042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1060/2124] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:54:55 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:54:55.332550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:55.357797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7849f0-e701-4a64-99af-4a84b69f3bf4 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:55.379619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:55.381448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.382819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.383330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.384764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.387761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.423305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1065/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:55.427954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:55.459426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdb6d-0104-4736-8e41-4b0d415353fa admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:55.468663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:55.470168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.475741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.479093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.480398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.495862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489563-b40e-4cfb-94da-729a2479752e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.541450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1061/2126] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:54:55.546704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:55.558942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915aad49-1355-4426-b8ab-bbee46b71bc1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:55.565443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:55.566142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.566427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.566721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] PATH_INFO: `/v3/projects/ffc5ce52ae574b82bda143e766bcec80/users/0d81b048f7e64c53894ad9e9ee33f518/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.591291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc5ce52ae574b82bda143e766bcec80, user_id=0d81b048f7e64c53894ad9e9ee33f518, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:55.595332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:55.620815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1066/2127] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:54:55 2022] PUT /identity/v3/projects/ffc5ce52ae574b82bda143e766bcec80/users/0d81b048f7e64c53894ad9e9ee33f518/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:55.623358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:55.624420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:55.625048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:55.719806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:54:55.729178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:55.997513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1062/2128] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:54:56.017624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:56.020223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:56.228045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946793-4c30-4574-acad-ac9c6feda873 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:56.237057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:56.237960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:56.238433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:56.238981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:56.278037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:56.280314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:56.305657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1067/2129] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:54:57.840944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:57.843192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:57.889876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7ae8f-c9d4-4d10-a15d-df46be03be2d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:57.913168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:57.915726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:57.916566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:57.916909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] PATH_INFO: `/v3/users/20d79e4c3e72404f85cbe78dbb1e3fde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:57.918990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d79e4c3e72404f85cbe78dbb1e3fde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:57.943195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:58.110511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d79e4c3e72404f85cbe78dbb1e3fde'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:58.115842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d79e4c3e72404f85cbe78dbb1e3fde'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:58.117308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d79e4c3e72404f85cbe78dbb1e3fde'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:58.155413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d79e4c3e72404f85cbe78dbb1e3fde'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:58.165105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:58.199184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d79e4c3e72404f85cbe78dbb1e3fde'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:58.248082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1063/2130] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:57 2022] DELETE /identity/v3/users/20d79e4c3e72404f85cbe78dbb1e3fde => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:54:58.808351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:58.854739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b9553-6137-41c9-93f0-6ac1a0680908 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:59.087572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:54:59.135034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:59.136147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:59.136632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:59.137140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] PATH_INFO: `/v3/projects/baab1921435a4554b2de19f97c41df5f` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:59.140034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baab1921435a4554b2de19f97c41df5f)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:59.143735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:59.353163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baab1921435a4554b2de19f97c41df5f', 'request_id': 'req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.365814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baab1921435a4554b2de19f97c41df5f', 'request_id': 'req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.384826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baab1921435a4554b2de19f97c41df5f', 'request_id': 'req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.424860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] The token cache is being invalidate because project baab1921435a4554b2de19f97c41df5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:54:59.425249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.426106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.428243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1068/2131] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:54:58 2022] DELETE /identity/v3/projects/baab1921435a4554b2de19f97c41df5f => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:54:59.433150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:54:59.436565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:54:59.460170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddc35-597a-41dd-98d3-a325f707ec26 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:54:59.636472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:54:59.641972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:54:59.641972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:54:59.641972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] PATH_INFO: `/v3/users/a6ab20d710d54250be2fd29f81f4dd2d` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:54:59.645932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ab20d710d54250be2fd29f81f4dd2d)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:54:59.655380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:54:59.823452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ab20d710d54250be2fd29f81f4dd2d'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.830435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ab20d710d54250be2fd29f81f4dd2d'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.832683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ab20d710d54250be2fd29f81f4dd2d'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.888406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ab20d710d54250be2fd29f81f4dd2d'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.937824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ab20d710d54250be2fd29f81f4dd2d'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:54:59.977268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1064/2132] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:54:59 2022] DELETE /identity/v3/users/a6ab20d710d54250be2fd29f81f4dd2d => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:00.828404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:00.830060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:00.853993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efc32-99c9-4bc4-b5b0-1416a81a69a1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:01.013683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:01.015346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:01.122088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:01.122884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:01.123253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:01.123575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] PATH_INFO: `/v3/users/1abc96453ab2428ab59ca3935666bfcc` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:01.132400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abc96453ab2428ab59ca3935666bfcc)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:01.135386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:01.304178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abc96453ab2428ab59ca3935666bfcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.305495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abc96453ab2428ab59ca3935666bfcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.323383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abc96453ab2428ab59ca3935666bfcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.325108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:01.344931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abc96453ab2428ab59ca3935666bfcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.362098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e84da-f8c7-4c2d-9189-2c2ea25dfa34 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:01.376791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:01.379727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a37a6d35-0d6b-4aa3-8346-e9cbb8c79682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abc96453ab2428ab59ca3935666bfcc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.409131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:01.411432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:01.414126 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1069/2133] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:00 2022] DELETE /identity/v3/users/1abc96453ab2428ab59ca3935666bfcc => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:01.418689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:01.423923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] PATH_INFO: `/v3/projects/1a41068bd73247eeaf090fc1878a46a8` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:01.431200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a41068bd73247eeaf090fc1878a46a8)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:01.440565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:01.443341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:01.443965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:01.444763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:01.516427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:01.521307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:01.668564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a41068bd73247eeaf090fc1878a46a8', 'request_id': 'req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.707762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a41068bd73247eeaf090fc1878a46a8', 'request_id': 'req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.710689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1070/2134] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:01.711780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:01.713409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:01.738521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a41068bd73247eeaf090fc1878a46a8', 'request_id': 'req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.798666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] The token cache is being invalidate because project 1a41068bd73247eeaf090fc1878a46a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:01.799505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.800820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:01.808945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1065/2135] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:01 2022] DELETE /identity/v3/projects/1a41068bd73247eeaf090fc1878a46a8 => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:01.812542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:01.814944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:01.889201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b16db-ab24-41ff-97bf-1f16d2642581 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:02.008942 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6a21-1c6f-4ba3-a34c-90f6f8a96d7c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:02.016275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:02.017318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.017318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.017792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.023386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:02.039525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:02.040367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.040774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.041891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.047430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:02.213323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:02.216351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd6ea-f414-4c81-903e-312440b6574e service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:02.221264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:02.224215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f12524-dc2b-4b4d-af61-b98df97ab8d5 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:02.257396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1066/2136] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:02.260534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1071/2137] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:02.262360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.263068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.263701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.309981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:02.314106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:02.351112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1067/2138] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:02.366523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752583a-e279-4041-9253-50495166dd3b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.367421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752583a-e279-4041-9253-50495166dd3b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.368417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752583a-e279-4041-9253-50495166dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.420755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:02.444526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b752583a-e279-4041-9253-50495166dd3b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:02.455883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b752583a-e279-4041-9253-50495166dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:02.519136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1072/2139] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:02.525065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:02.557346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48399b4-9cc7-4bf8-a106-f1ce26e9c363 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:02.578808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:02.580987 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.582141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.582491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.587861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:02.588522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:02.605553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:02.648629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1068/2140] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:02.652199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752583a-e279-4041-9253-50495166dd3b None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:02.698331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752583a-e279-4041-9253-50495166dd3b None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:02.718758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:02.720252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:02.720417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:02.720809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:02.724632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:02.728218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:02.853469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1073/2141] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:02 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:02.855604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:02.902736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449d5c2-4cf8-4e1d-a147-699c5383e7cb admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:02.944505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:03.107426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:03.108480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:03.108905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:03.109365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:03.110618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:03.114345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:03.170247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1069/2142] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:02 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:03.173029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:03.215974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678335f-333d-4b20-ac97-8345bb541981 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:03.230532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:03.231556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:03.235496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:03.236011 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:03.238309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:03.243117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:03.277865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1074/2143] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:03.282510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:03.297657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d28a5-65f3-496d-b0a6-d650dac92d2c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:03.313268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:03.313913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:03.314222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:03.314494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:03.315696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:03.321024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77431d0d-9546-4a3e-8746-d4f4cd0a2e85 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:03.349868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1070/2144] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:03.354365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:03.372392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26d81c-b281-4da3-a801-bc4809dad585 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:03.386874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:03.387608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:03.387891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:03.388164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] PATH_INFO: `/v3/projects/f4168fd9e6b54e0f8aacc358953c0b12/users/00e14700223f4bb692bd6433a40d0b96/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:03.413752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4168fd9e6b54e0f8aacc358953c0b12, user_id=00e14700223f4bb692bd6433a40d0b96, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:03.417713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:03.442744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1075/2145] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:03 2022] PUT /identity/v3/projects/f4168fd9e6b54e0f8aacc358953c0b12/users/00e14700223f4bb692bd6433a40d0b96/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:03.455595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:03.457098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:03.457691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:03.547150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] MFA Rules not processed for user `00e14700223f4bb692bd6433a40d0b96`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:03.564975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:03.900038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1071/2146] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:03.906676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:03.911024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:03.924316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:03.929882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:04.193201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebb21b-3e58-4ab4-9187-af8d8cd4b1f8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:04.205179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:04.210452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:04.215388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:04.216259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:04.249664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51f2e5-4f8d-4db6-be62-54bed5f40818 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:04.253411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:04.257163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024937a-ef0a-4894-853d-f370e53fc189 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:04.283695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:04.284853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:04.285474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:04.285916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] PATH_INFO: `/v3/projects/350e5a298325458bb3657e02c7de30c2` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:04.292696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=350e5a298325458bb3657e02c7de30c2)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:04.296220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:04.300513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1072/2147] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:04.527950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350e5a298325458bb3657e02c7de30c2', 'request_id': 'req-6e86d239-a9bd-4954-8f43-71fb21ed8aee'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:04.563967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350e5a298325458bb3657e02c7de30c2', 'request_id': 'req-6e86d239-a9bd-4954-8f43-71fb21ed8aee'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:04.580707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350e5a298325458bb3657e02c7de30c2', 'request_id': 'req-6e86d239-a9bd-4954-8f43-71fb21ed8aee'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:04.599167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] The token cache is being invalidate because project 350e5a298325458bb3657e02c7de30c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:04.599557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:04.600252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6e86d239-a9bd-4954-8f43-71fb21ed8aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:04.602909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1076/2148] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:03 2022] DELETE /identity/v3/projects/350e5a298325458bb3657e02c7de30c2 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:04.610228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:04.610685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:04.611176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:04.699770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:04.701967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:04.868128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:05.045165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.045480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.045771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.057457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1073/2149] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:05.086965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:05.089658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:05.115117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.126100 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1077/2150] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:05.128819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7b816-4689-493b-a9d8-ea39702f3284 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.145557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.146853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.147378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.147886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.148761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:05.149277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.151257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.188867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.195526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1074/2151] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:55:05.201102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8d73-7218-4473-b150-94a8bb941250 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.210922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.211997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.212459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.213806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.214794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.217655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.264108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1078/2152] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:05 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:05.271787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.307822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50425b6-32a0-4e60-9222-d0e4ec87a0f9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.536877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.537697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.538072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.539080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.539080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.541328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.581985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.590429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:05.596791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1075/2153] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:05 2022] POST /identity/v3/users => generated 286 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:05.599727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340e05a-dca7-4063-9646-e061fcea1cf7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.622639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.623890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.624624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.628843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.630014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.634052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.661399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.672915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3203bf5-619c-46da-9d33-681a085dd4b8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.676360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1079/2154] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:05.689692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.690638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.691102 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.691450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.692841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.696175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831863e-f954-4281-8274-11e419d13a8f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.744410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1076/2155] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:05.748851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:05.759869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fc9ef-7805-4400-b638-ae11afc96e0d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:05.777006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:05.778465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.779711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.780120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] PATH_INFO: `/v3/projects/3258030da65a44ecbb3751ddaac98549/users/79eaf31d633d4bfd9d652a1d69db3961/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.838272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3258030da65a44ecbb3751ddaac98549, user_id=79eaf31d633d4bfd9d652a1d69db3961, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:05.841287 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:05.883412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1080/2156] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:05 2022] PUT /identity/v3/projects/3258030da65a44ecbb3751ddaac98549/users/79eaf31d633d4bfd9d652a1d69db3961/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:05.885150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:05.885451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:05.885769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:05.950190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] MFA Rules not processed for user `79eaf31d633d4bfd9d652a1d69db3961`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:05.952091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:06.135376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1077/2157] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:06.149884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:06.152430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:06.284283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b18fe-58bf-45d8-a0fb-5c4562c5946a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:06.290227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:06.291212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:06.291642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:06.292035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:06.309605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:06.311644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:06.328484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1081/2158] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:06.456892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:06.618513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:06.792088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7941b-3256-475f-9697-2b42d03f36eb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:06.808320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:06.809124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:06.809298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:06.809573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] PATH_INFO: `/v3/users/79eaf31d633d4bfd9d652a1d69db3961` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:06.818607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79eaf31d633d4bfd9d652a1d69db3961)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:06.830770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:06.957495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:06.961950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:07.043931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d44899-601a-41fe-acf2-66ac78718b7d service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:07.072903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:07.074513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:07.074658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79eaf31d633d4bfd9d652a1d69db3961'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.074825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:07.074825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] PATH_INFO: `/v3/users/8c444d45b8194f9ca4c46a55570a7717` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:07.076023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79eaf31d633d4bfd9d652a1d69db3961'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.076249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c444d45b8194f9ca4c46a55570a7717)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:07.078840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:07.081040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79eaf31d633d4bfd9d652a1d69db3961'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.102487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79eaf31d633d4bfd9d652a1d69db3961'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.171212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-65c06609-5ddc-435e-9cab-a3a791b7c006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79eaf31d633d4bfd9d652a1d69db3961'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.269566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1078/2159] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:06 2022] DELETE /identity/v3/users/79eaf31d633d4bfd9d652a1d69db3961 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:07.277649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:07.278612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:07.280593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:07.294487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c444d45b8194f9ca4c46a55570a7717'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.296725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c444d45b8194f9ca4c46a55570a7717'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.354421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c444d45b8194f9ca4c46a55570a7717'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.417598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c444d45b8194f9ca4c46a55570a7717'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.445067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:07.462661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:07.550482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c06908e8-df10-4764-965c-a9641360c2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c444d45b8194f9ca4c46a55570a7717'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:07.586773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1082/2160] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:06 2022] DELETE /identity/v3/users/8c444d45b8194f9ca4c46a55570a7717 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:07.594335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:07.596545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:07.597242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:07.686877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:07.688935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:07.717596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1079/2161] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:07.728970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:07.732017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:07.843481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:07.877717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:07.889904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1083/2162] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:07.907265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b5c86-8b07-4361-bfd4-7c386fad3901 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:07.909515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:07.991253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:07.992019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:07.992359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:07.996811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:08.012170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:08.016593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04526a-3e8b-47f7-9e74-d0b9da260f64 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:08.057569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1080/2163] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:08.090019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735b634-52b7-45e2-9dd6-07f917b87b03 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:08.099752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:08.101646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:08.102118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:08.103314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:08.141938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:08.144741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:08.189333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1084/2164] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:08.440138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:08.440443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:08.440777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:08.529970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] MFA Rules not processed for user `0d81b048f7e64c53894ad9e9ee33f518`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:08.533024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:08.878018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1081/2165] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:08.900373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:08.905402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:09.203159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:09.204649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:09.211472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdba53-de77-4e8b-bddc-697b336e8f05 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:09.219877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:09.220697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:09.220897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:09.221142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:09.231735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:09.234032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:09.247471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfaed5-6258-4713-90bd-da04449cb586 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:09.254139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:09.255351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:09.255899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:09.256454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] PATH_INFO: `/v3/projects/f7f4f3447942486b9e5019a43b4e5659` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:09.258320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f4f3447942486b9e5019a43b4e5659)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:09.260977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:09.268181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1085/2166] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:09.381747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f4f3447942486b9e5019a43b4e5659', 'request_id': 'req-27467659-5de4-4131-9675-ae492e5c1372'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.418006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f4f3447942486b9e5019a43b4e5659', 'request_id': 'req-27467659-5de4-4131-9675-ae492e5c1372'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.451010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:09.454581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f4f3447942486b9e5019a43b4e5659', 'request_id': 'req-27467659-5de4-4131-9675-ae492e5c1372'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.455134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:09.495838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] The token cache is being invalidate because project f7f4f3447942486b9e5019a43b4e5659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:09.496526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.498255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.504555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1082/2167] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:09 2022] DELETE /identity/v3/projects/f7f4f3447942486b9e5019a43b4e5659 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:09.510368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:09.514711 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:09.554171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebcb3b-627e-4c01-a024-e78e890bc47c service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:09.586010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:09.666023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:09.667096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:09.667415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:09.667692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] PATH_INFO: `/v3/projects/3258030da65a44ecbb3751ddaac98549` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:09.669250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3258030da65a44ecbb3751ddaac98549)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:09.671412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:09.687282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27467659-5de4-4131-9675-ae492e5c1372 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:09.693084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:09.693962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:09.694240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:09.694712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:09.696834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] PATH_INFO: `/v3/users/4d1e49f684474b8b9d3fdb89915d34a8` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:09.698389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1e49f684474b8b9d3fdb89915d34a8)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:09.701235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:09.808883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3258030da65a44ecbb3751ddaac98549', 'request_id': 'req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.818027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3258030da65a44ecbb3751ddaac98549', 'request_id': 'req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.837896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3258030da65a44ecbb3751ddaac98549', 'request_id': 'req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.861627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1e49f684474b8b9d3fdb89915d34a8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.862835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1e49f684474b8b9d3fdb89915d34a8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.864160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1e49f684474b8b9d3fdb89915d34a8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.889746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1e49f684474b8b9d3fdb89915d34a8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.899641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] The token cache is being invalidate because project 3258030da65a44ecbb3751ddaac98549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:09.899951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.904097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-50ccc7dc-d9de-40f3-a28b-d81d0ab0800e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.914074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1086/2168] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:09 2022] DELETE /identity/v3/projects/3258030da65a44ecbb3751ddaac98549 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:09.922290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:09.922682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:09.922972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:09.923304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33ca4fcd-47a0-4a6e-9e62-7e7c1ca63464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1e49f684474b8b9d3fdb89915d34a8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:09.972607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1083/2169] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:09 2022] DELETE /identity/v3/users/4d1e49f684474b8b9d3fdb89915d34a8 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:09.974156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:09.975620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:10.211356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1087/2170] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:10.225485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:10.225830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:10.226157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:10.292134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:10.293315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:10.333750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1084/2171] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:10.342927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:10.363488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad087f9-14bf-4848-9ede-d55e9ab38306 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:10.377331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:10.379825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:10.381460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:10.381460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:10.382909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:10.383419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:10.385937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:10.442850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:10.453882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1088/2172] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:10.473322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c24f-b6f1-4081-85e2-2ae13a770d0e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:10.489471 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:10.495721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:10.496333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:10.497685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:10.509001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:10.520326 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:10.688661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1085/2173] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:10 2022] POST /identity/v3/projects => generated 391 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:10.694795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:10.738617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac4d53-254b-4b55-aca5-1d90fb975990 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:10.908351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:10.909855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:10.910990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:10.913193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:10.916845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:10.925984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:10.996828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.000902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1089/2174] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:10 2022] POST /identity/v3/users => generated 296 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:11.002449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.018548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:11.043416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d01c8-512f-4308-9898-9fcfcf9c430c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:11.051081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:11.052096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.052629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.053108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.054035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:11.063354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb7e7-8623-43bd-8053-0a6d486f027f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:11.096377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:11.101684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:11.103410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.104543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.109847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1086/2175] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:11.110761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.110987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] PATH_INFO: `/v3/projects/57ef4ad6675d4e7e97d120f17e0be345` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.114227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ef4ad6675d4e7e97d120f17e0be345)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:11.120613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:11.140327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ab669-3e5e-4f60-8ca7-2996371602dc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:11.168115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:11.169883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:11.170859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.174801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.176470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.178641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:11.180404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:11.214903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1087/2176] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:11.221700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.245071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04760-c33e-4ecf-8084-03af6ee1fbc3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:11.301880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ef4ad6675d4e7e97d120f17e0be345', 'request_id': 'req-abccf635-157f-4c55-a3bf-c9cda0979b73'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.328833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ef4ad6675d4e7e97d120f17e0be345', 'request_id': 'req-abccf635-157f-4c55-a3bf-c9cda0979b73'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.344682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ef4ad6675d4e7e97d120f17e0be345', 'request_id': 'req-abccf635-157f-4c55-a3bf-c9cda0979b73'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.368952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:11.412671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] The token cache is being invalidate because project 57ef4ad6675d4e7e97d120f17e0be345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:11.414680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.416238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.421170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1090/2177] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:11 2022] DELETE /identity/v3/projects/57ef4ad6675d4e7e97d120f17e0be345 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:11.426609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:11.426609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.426609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.426609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] PATH_INFO: `/v3/projects/008bda799c4a4110ba61171e4f2f9890/users/13b8d70f2288460e8ee4436379c619d7/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.429455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.432274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:11.464448 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccf635-157f-4c55-a3bf-c9cda0979b73 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:11.472482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008bda799c4a4110ba61171e4f2f9890, user_id=13b8d70f2288460e8ee4436379c619d7, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:11.473795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:11.474670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.475117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.475588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] PATH_INFO: `/v3/users/8bcf888b7cd4493594dcf5ec4364f150` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.477043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bcf888b7cd4493594dcf5ec4364f150)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:11.479021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e1ed-5b6b-48ff-b36e-44e2e603dcc0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:11.479643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:11.527255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1088/2178] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:11 2022] PUT /identity/v3/projects/008bda799c4a4110ba61171e4f2f9890/users/13b8d70f2288460e8ee4436379c619d7/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:11.537787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:11.537787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:11.537787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:11.604757 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] MFA Rules not processed for user `13b8d70f2288460e8ee4436379c619d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:11.609620 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:11.688644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bcf888b7cd4493594dcf5ec4364f150'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.693879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bcf888b7cd4493594dcf5ec4364f150'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.752028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bcf888b7cd4493594dcf5ec4364f150'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.785731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bcf888b7cd4493594dcf5ec4364f150'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.826639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bcf888b7cd4493594dcf5ec4364f150'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:11.847083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1091/2179] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:11 2022] DELETE /identity/v3/users/8bcf888b7cd4493594dcf5ec4364f150 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:11.885738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1089/2180] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:11.973793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:11.977471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:12.279392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79220f-79c0-445c-ab35-b76a5044d5ff admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:12.297058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:12.298541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:12.299067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:12.302993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:12.455516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:12.596264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:12.599753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9a080-6206-4fce-870e-f02ab79da2a0 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:12.699253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1092/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:13.373036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:13.375140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:13.615295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24cbfc-75bf-4a61-bd79-4b7e8729d27a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:14.084144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb7940-ce07-4ea5-9c59-d21537ecd728 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:14.084838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb7940-ce07-4ea5-9c59-d21537ecd728 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:14.086112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb7940-ce07-4ea5-9c59-d21537ecd728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:14.104959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:14.106491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:14.107401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:14.109671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] PATH_INFO: `/v3/projects/cb981d74a4c04fc890148ed7ed4b962c` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:14.113118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb981d74a4c04fc890148ed7ed4b962c)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:14.128693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:14.147768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:14.171150 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f6cb7940-ce07-4ea5-9c59-d21537ecd728 None None] MFA Rules not processed for user `0d81b048f7e64c53894ad9e9ee33f518`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:14.177986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f6cb7940-ce07-4ea5-9c59-d21537ecd728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:14.279778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb981d74a4c04fc890148ed7ed4b962c', 'request_id': 'req-d48159c5-052f-4a77-b401-58eb258aa6af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:14.318047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb981d74a4c04fc890148ed7ed4b962c', 'request_id': 'req-d48159c5-052f-4a77-b401-58eb258aa6af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:14.376040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb981d74a4c04fc890148ed7ed4b962c', 'request_id': 'req-d48159c5-052f-4a77-b401-58eb258aa6af'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:14.414483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] The token cache is being invalidate because project cb981d74a4c04fc890148ed7ed4b962c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:14.415104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:14.416171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d48159c5-052f-4a77-b401-58eb258aa6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:14.420483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1090/2182] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:13 2022] DELETE /identity/v3/projects/cb981d74a4c04fc890148ed7ed4b962c => generated 0 bytes in 1048 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:14.430059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:14.430931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:14.431592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:14.473625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:14.476127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:14.489958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1093/2183] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:14.726118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1091/2184] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:14.741746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:14.747898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:14.749683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:14.941151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:14.946439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:14.993877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1094/2185] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:14.996002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:15.026950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62761f9b-9f5a-4c89-adff-6302afc45f2a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:15.062441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:15.063754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:15.064695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:15.065108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:15.067424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:15.068347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:15.072067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:15.100111 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1092/2186] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:15.105651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:15.129656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484170f1-e06b-4b8a-a62e-084988c2d654 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:15.140217 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:15.142940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:15.143198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:15.145834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:15.146776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:15.152333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:15.270012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1095/2187] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:15 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:15.275300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:15.308749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04580321-e811-42c7-a7c3-63a77c42d799 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:15.645646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:15.646703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:15.646860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:15.647100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:15.648539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:15.650050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:15.687361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1093/2188] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:15 2022] POST /identity/v3/users => generated 284 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:15.748360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:15.833167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2256e2-ae40-40b7-ac5f-9207626101eb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:15.855036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:15.856680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:15.857141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:15.857685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:15.868053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:15.880933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:15.919788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1096/2189] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:15.921255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:15.943656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c448-ed89-4908-bb3f-226467645222 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:15.959090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:15.959765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:15.960044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:15.960314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:15.960993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:15.962908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35483810-a6d7-48b5-98b7-df5230e32c74 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:15.984962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:16.015283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:16.029971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1094/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:16.030868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18110d-ca3d-47c1-aa8e-fb6a74d75b18 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:16.042539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:16.043291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.043574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.043846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] PATH_INFO: `/v3/projects/32a568188c1c4cc9b57250ce7c07f831/users/9da7dd14d6a6457c8ce90827245f30f5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.081376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a568188c1c4cc9b57250ce7c07f831, user_id=9da7dd14d6a6457c8ce90827245f30f5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:16.091847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:16.140830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1097/2191] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:16 2022] PUT /identity/v3/projects/32a568188c1c4cc9b57250ce7c07f831/users/9da7dd14d6a6457c8ce90827245f30f5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:16.175864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.176539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.177090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.221655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] MFA Rules not processed for user `9da7dd14d6a6457c8ce90827245f30f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:16.223668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:16.355262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1095/2192] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:16.363372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:16.532736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ae99-4c14-42ea-bb8b-c2c8e5be74fc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:16.541388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:16.542629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.543200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.543937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.545214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:16.549531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:16.578294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:16.597556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1098/2193] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:16 2022] POST /identity/v3/projects => generated 367 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:16.603060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:16.624327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce07a-1030-48c3-8a39-1743fb0069d6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:16.759015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:16.760485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.761151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.761746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.762819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:16.769125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:16.828339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1096/2194] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:16 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:16.829906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:16.856948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e244ca-b989-4720-8ab7-9fb62cfd0e02 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:16.874545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:16.880562 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.880993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.881417 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.882526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:16.888979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1382a4e-b60a-42bf-8e31-c5fa8d1bc789 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:16.931622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:16.932096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1099/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:16.945305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.947450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f90a3-7746-4c5f-ac2c-d0b3da434d23 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:16.948281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.948625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:16.977787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:16.979968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:16.979968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:16.983728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] PATH_INFO: `/v3/projects/476889f44b294a828d6a6d58b7412622/users/2045ca34767d4a678c13ebd1bebac924/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:17.001123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] MFA Rules not processed for user `e76b3353ce4b4361872ed6dba9f895d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:17.006062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:17.040271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476889f44b294a828d6a6d58b7412622, user_id=2045ca34767d4a678c13ebd1bebac924, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:17.046982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:17.090986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1097/2196] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:16 2022] PUT /identity/v3/projects/476889f44b294a828d6a6d58b7412622/users/2045ca34767d4a678c13ebd1bebac924/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:17.097617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:17.244219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1100/2197] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:17.390451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:17.418867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acdb0e-c4d4-4f2e-97bf-39ff34379307 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:17.440950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:17.441524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:17.441810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:17.442092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:17.442778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:17.444585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491fd22-bf48-4b48-a263-9b958e46c839 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:17.468018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:17.468528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1098/2198] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:17.484083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db292b-2b95-4563-8e68-5536f2ac4baf None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:17.493366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:17.495069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:17.495653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:17.496267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] PATH_INFO: `/v3/projects/476889f44b294a828d6a6d58b7412622/users/2045ca34767d4a678c13ebd1bebac924/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:17.499198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476889f44b294a828d6a6d58b7412622, user_id=2045ca34767d4a678c13ebd1bebac924, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:17.524964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:17.573188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1101/2199] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:17 2022] PUT /identity/v3/projects/476889f44b294a828d6a6d58b7412622/users/2045ca34767d4a678c13ebd1bebac924/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:17.584816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:17.585127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:17.585374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:17.634115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] MFA Rules not processed for user `2045ca34767d4a678c13ebd1bebac924`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:17.635647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:17.783257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1099/2200] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:17.827043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:17.830210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:17.969074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5854a-37fd-476f-8a67-39492e12bdf0 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:17.974219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:17.975193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:17.975784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:17.976154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:18.001449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:18.002996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:18.032822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1102/2201] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:18.182580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:18.196421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd59bd9-4887-40ba-b433-5fff63b54510 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:18.203450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:18.206787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:18.206787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:18.206787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:18.307882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:18.309698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43d1a7-8ace-4ea2-ba2e-16d3ecd76283 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:18.321769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1100/2202] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:19.508695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:19.510324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:19.616581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51622c-7d98-4a57-80ec-231b11eec1d1 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:19.621989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:19.622720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:19.623113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:19.623419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] PATH_INFO: `/v3/users/00e14700223f4bb692bd6433a40d0b96` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:19.624934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e14700223f4bb692bd6433a40d0b96)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:19.627496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:19.670685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:19.705811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e14700223f4bb692bd6433a40d0b96'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:19.706468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e14700223f4bb692bd6433a40d0b96'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:19.717549 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e14700223f4bb692bd6433a40d0b96'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:19.726995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e14700223f4bb692bd6433a40d0b96'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:19.740636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e14700223f4bb692bd6433a40d0b96'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:19.747528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1103/2203] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:19 2022] DELETE /identity/v3/users/00e14700223f4bb692bd6433a40d0b96 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:19.752256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:19.752876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:19.752876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:19.774509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:19.776139 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:19.842971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1101/2204] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:19.854183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:19.855673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:19.923996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f825-4514-4071-aba6-9fc8f56d4d03 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:19.942271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:19.942913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:19.943193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:19.943503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:19.960573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:19.961563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db8c7-06c9-428b-890c-220db51943ba service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:19.973058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1104/2205] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:20.326630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:20.328963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:20.352578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344e75-2144-47b4-bfca-e839a5a60aba None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:20.360910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:20.361245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.361475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.361781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] PATH_INFO: `/v3/projects/f4168fd9e6b54e0f8aacc358953c0b12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.363043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4168fd9e6b54e0f8aacc358953c0b12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:20.365695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:20.379884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:20.437645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4168fd9e6b54e0f8aacc358953c0b12', 'request_id': 'req-36b24332-8b7d-4a1a-823d-e0011ba8d293'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:20.446008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4168fd9e6b54e0f8aacc358953c0b12', 'request_id': 'req-36b24332-8b7d-4a1a-823d-e0011ba8d293'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:20.457614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4168fd9e6b54e0f8aacc358953c0b12', 'request_id': 'req-36b24332-8b7d-4a1a-823d-e0011ba8d293'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:20.478580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] The token cache is being invalidate because project f4168fd9e6b54e0f8aacc358953c0b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:20.479142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:20.480301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36b24332-8b7d-4a1a-823d-e0011ba8d293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:20.482068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1102/2206] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:20 2022] DELETE /identity/v3/projects/f4168fd9e6b54e0f8aacc358953c0b12 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:20.488800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.490309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.490309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.523511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:20.524907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:20.604542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1105/2207] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:20.617724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.618010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.618264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.643507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:20.644922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:20.653875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1103/2208] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:20.658743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:20.671052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b443c-53bc-4461-9207-615add18c8c2 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:20.678329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:20.679391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.679818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.680232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.681234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:20.681542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:20.685388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:20.710614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1106/2209] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:20.715970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:20.734507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0cf16-41cf-4073-aaa7-d5a5cc4c5cc3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:20.742584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:20.743673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.744153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.744652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.745881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:20.749744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:20.793500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1104/2210] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:20 2022] POST /identity/v3/projects => generated 361 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:20.799631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:20.821216 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62417f-2c13-4d72-9f09-529a7b7e0896 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:20.868109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:20.910954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:20.911633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.911914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.912182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.912866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:20.914866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:20.942655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1107/2211] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:20 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:20.947345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:20.960167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869dc42-8fde-402d-ae0e-112a963656d3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:20.965934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:20.966601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:20.966876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:20.967147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:20.967932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:20.969604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:20.988578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1105/2212] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:20.994685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:21.007149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9597-08b6-4eb5-bd92-62635b634863 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:21.013347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:21.014039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.014330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.014620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.015326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:21.017027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd18f3f-9331-4869-8d59-418588e976a9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:21.032808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1108/2213] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:21.038334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:21.051486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e48bf9-5559-4969-b2fb-8b525971f082 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:21.058313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:21.059020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.059354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.059610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] PATH_INFO: `/v3/projects/1c13492e364d46e7a1de249ea6c0e795/users/9a119ea891994c7c9bc942639b85a7dc/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.079994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c13492e364d46e7a1de249ea6c0e795, user_id=9a119ea891994c7c9bc942639b85a7dc, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:21.082964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:21.100695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1106/2214] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:21 2022] PUT /identity/v3/projects/1c13492e364d46e7a1de249ea6c0e795/users/9a119ea891994c7c9bc942639b85a7dc/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:21.112962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.113583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.114010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.150992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] MFA Rules not processed for user `9a119ea891994c7c9bc942639b85a7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:21.153138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:21.233736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1109/2215] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:21.275745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:21.277916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:21.371435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2361-e4c4-40cb-80e8-e650b41482c2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:21.378847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:21.379935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.380364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.380772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.395585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:21.397953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e9fdc-0665-48d4-83a0-a07e456cd197 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:21.411931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1107/2216] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:21.432512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:21.560194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d288c9a-f5be-47a3-8e1f-5096408fbd25 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:21.569360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:21.570392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.571010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.571437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] PATH_INFO: `/v3/users/9a119ea891994c7c9bc942639b85a7dc` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.579862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a119ea891994c7c9bc942639b85a7dc)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:21.579862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:21.635921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:21.649760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a119ea891994c7c9bc942639b85a7dc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:21.650796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a119ea891994c7c9bc942639b85a7dc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:21.672151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a119ea891994c7c9bc942639b85a7dc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:21.686356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a119ea891994c7c9bc942639b85a7dc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:21.723756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a119ea891994c7c9bc942639b85a7dc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:21.735346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1110/2217] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:21 2022] DELETE /identity/v3/users/9a119ea891994c7c9bc942639b85a7dc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:21.737251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:21.737527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:21.737791 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:21.770215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:21.772291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:21.847531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:21.905480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1108/2218] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:21.923158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:21.925406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:22.057776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6e17a-e1dd-4799-87b9-cfffe4bea58f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:22.076796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:22.077446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:22.077760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:22.078027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:22.090146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:22.096970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:22.115034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1111/2219] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:22.498950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:22.502043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:22.549142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513608d-69ad-40e6-8e37-16fe8627aaa8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:22.562080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:22.562989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:22.563759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:22.564647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] PATH_INFO: `/v3/projects/1c13492e364d46e7a1de249ea6c0e795` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:22.570027 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c13492e364d46e7a1de249ea6c0e795)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:22.574404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:22.692382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c13492e364d46e7a1de249ea6c0e795', 'request_id': 'req-1257a667-b203-47d1-9f01-c092a5370b1f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:22.703983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c13492e364d46e7a1de249ea6c0e795', 'request_id': 'req-1257a667-b203-47d1-9f01-c092a5370b1f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:22.734172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c13492e364d46e7a1de249ea6c0e795', 'request_id': 'req-1257a667-b203-47d1-9f01-c092a5370b1f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:22.787808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] The token cache is being invalidate because project 1c13492e364d46e7a1de249ea6c0e795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:22.788387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:22.789811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1257a667-b203-47d1-9f01-c092a5370b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:22.792889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1109/2220] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:22 2022] DELETE /identity/v3/projects/1c13492e364d46e7a1de249ea6c0e795 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:24.159178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:24.163158 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:24.188638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bec6d-7be8-48f2-b290-5517fc635900 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:24.286075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:24.286890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:24.287171 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:24.287629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] PATH_INFO: `/v3/users/9da7dd14d6a6457c8ce90827245f30f5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:24.289227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da7dd14d6a6457c8ce90827245f30f5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:24.292203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:24.359095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da7dd14d6a6457c8ce90827245f30f5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:24.360829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da7dd14d6a6457c8ce90827245f30f5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:24.369600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da7dd14d6a6457c8ce90827245f30f5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:24.375302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:24.382397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da7dd14d6a6457c8ce90827245f30f5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:24.403617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da7dd14d6a6457c8ce90827245f30f5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:24.411191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1112/2221] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:24 2022] DELETE /identity/v3/users/9da7dd14d6a6457c8ce90827245f30f5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:24.418136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:24.419064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:24.419476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:24.444749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:24.446174 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:24.536810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1110/2222] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:24.544230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:24.545991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:24.635268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436a9-f705-4019-a090-0ed0139019fe admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:24.639683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:24.640623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:24.641069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:24.641503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:24.652891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:24.654398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:24.667620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1113/2223] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:25.170492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:25.173600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:25.195439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9982217-80c0-4056-ba6c-159331880fd8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:25.200885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:25.201557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:25.201835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:25.202106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] PATH_INFO: `/v3/projects/32a568188c1c4cc9b57250ce7c07f831` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:25.203405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a568188c1c4cc9b57250ce7c07f831)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:25.205570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:25.297126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a568188c1c4cc9b57250ce7c07f831', 'request_id': 'req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.311475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a568188c1c4cc9b57250ce7c07f831', 'request_id': 'req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.317006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:25.327005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a568188c1c4cc9b57250ce7c07f831', 'request_id': 'req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.347089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] The token cache is being invalidate because project 32a568188c1c4cc9b57250ce7c07f831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:25.347462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.349153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.356342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1111/2224] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:25 2022] DELETE /identity/v3/projects/32a568188c1c4cc9b57250ce7c07f831 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:25.357758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:25.368498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5386-a321-45ed-b475-737ee9cc0b9a service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:25.453110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:25.453790 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:25.454105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:25.454369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] PATH_INFO: `/v3/users/2045ca34767d4a678c13ebd1bebac924` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:25.455966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2045ca34767d4a678c13ebd1bebac924)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:25.458040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:25.534342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2045ca34767d4a678c13ebd1bebac924'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.535050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2045ca34767d4a678c13ebd1bebac924'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.543974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2045ca34767d4a678c13ebd1bebac924'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.553530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2045ca34767d4a678c13ebd1bebac924'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.568661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-226936ab-bcca-4cf1-ae2c-6f50f0628d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2045ca34767d4a678c13ebd1bebac924'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:25.571658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:25.578366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1114/2225] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:25 2022] DELETE /identity/v3/users/2045ca34767d4a678c13ebd1bebac924 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:26.037957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.040572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:26.077030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8f73-d4ee-446b-8da0-98ff5f25d55f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.158862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.159689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.159985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.160407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] PATH_INFO: `/v3/projects/476889f44b294a828d6a6d58b7412622` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.161931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=476889f44b294a828d6a6d58b7412622)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.164157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.224292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476889f44b294a828d6a6d58b7412622', 'request_id': 'req-0e2e1951-e904-4296-8ea2-34a49cc59b7b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:26.232501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476889f44b294a828d6a6d58b7412622', 'request_id': 'req-0e2e1951-e904-4296-8ea2-34a49cc59b7b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:26.243249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476889f44b294a828d6a6d58b7412622', 'request_id': 'req-0e2e1951-e904-4296-8ea2-34a49cc59b7b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:26.257350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] The token cache is being invalidate because project 476889f44b294a828d6a6d58b7412622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:26.257641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:26.258340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0e2e1951-e904-4296-8ea2-34a49cc59b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:26.260052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1112/2226] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:26 2022] DELETE /identity/v3/projects/476889f44b294a828d6a6d58b7412622 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:26.265768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.266226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.266532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.290114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:26.291399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:26.383337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1115/2227] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:26.389079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.389578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.390100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.415390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:26.416634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:26.425897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1113/2228] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:26.431077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.442649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478eae9-e502-4cd5-acf5-1dd392d70432 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.447728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.448371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.448653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.448913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.449596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:26.449912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.451680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.465953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1116/2229] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:26.471411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.482519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384aecd4-ecca-4159-8bf6-58e43f3a16a0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.488183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.489222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.489707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.490183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.491250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.493518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.526806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1114/2230] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:26.532015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.544471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe627a-636c-4a2d-af15-90feedd4c1aa admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.623427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.624090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.624365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.624632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.625290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.627894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.656687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1117/2231] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:26.662633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.676656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e373a13-45cf-4b4b-b648-95bd13bbdc7a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.682389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.683079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.683404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.683672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.684571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.686381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.699576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1115/2232] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:26.704323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.715640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90e83d-1623-424a-9ba6-7fd071942c9c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.720855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.721522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.721800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.722064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.722713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.724471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02312f-4979-4b59-ab8d-86653be71ba6 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.740936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1118/2233] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:26.746871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.763341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00df9f2-ea91-4eb8-9dae-8180af12d670 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:26.769226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:26.769922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.770195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.770465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] PATH_INFO: `/v3/projects/3b7883c9328040839a7dd086474442fd/users/0b83183a946e4d22b077eb03e41428b5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.789121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7883c9328040839a7dd086474442fd, user_id=0b83183a946e4d22b077eb03e41428b5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:26.792952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:26.809391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1116/2234] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:26 2022] PUT /identity/v3/projects/3b7883c9328040839a7dd086474442fd/users/0b83183a946e4d22b077eb03e41428b5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:26.820354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:26.820687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:26.821074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:26.855266 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] MFA Rules not processed for user `0b83183a946e4d22b077eb03e41428b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:26.856516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:26.920211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:26.932302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1119/2235] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:26.937191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:26.960513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:27.033667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7aaee-0704-48ea-9c0a-38bffc40c628 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.039647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.040462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.040615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.040817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.041770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.044005 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.084909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1117/2236] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:26 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:27.090298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.103984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aab92-94a4-4435-9d12-cfde6146b385 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.206859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.208283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.208671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.209007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.209737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.212048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.237174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1120/2237] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:27 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:27.243094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.256681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c22314-6cf9-421f-b92f-e422d8f57969 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.262124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.262922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.263339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.263670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.264414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.266388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.280526 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1118/2238] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:27.285054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.297650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd810f-b9ab-4a7d-9074-7fbba1373bd9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.302774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.303491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.303761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.304031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] PATH_INFO: `/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe/users/cb010201ec494d69b31137624da221fc/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.324906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93aaafcc874c4ce7b5f1a1645df645fe, user_id=cb010201ec494d69b31137624da221fc, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.328090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.347244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1121/2239] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:27 2022] PUT /identity/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe/users/cb010201ec494d69b31137624da221fc/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:27.351654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.365104 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0dee5-d450-422c-b1b5-8e5dfdec638f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.490530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.491560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.492062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.492497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.493648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.496862 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1aad0-d813-414e-b647-deea36a16012 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.519271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1119/2240] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:27.524666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.545243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccb6a4-d3c6-440f-9e3d-dfe83599ac4f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.553977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.555181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.555851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.556412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] PATH_INFO: `/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe/users/cb010201ec494d69b31137624da221fc/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.560367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93aaafcc874c4ce7b5f1a1645df645fe, user_id=cb010201ec494d69b31137624da221fc, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:27.567055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:27.588359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1122/2241] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:27 2022] PUT /identity/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe/users/cb010201ec494d69b31137624da221fc/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:27.601034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.601398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.601813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:27.645389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] MFA Rules not processed for user `cb010201ec494d69b31137624da221fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:27.647676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:27.809271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1120/2242] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:27.829335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:27.832841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:27.972886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab05b2-c1f5-4b69-b094-b0cf2d8813e5 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:27.979522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:27.980887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:27.981057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:27.981616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:28.000964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:28.005817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:28.034299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1123/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:28.208981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:28.236043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe22da-8ab5-419b-b36b-9f6339f54191 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:28.256110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:28.256110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:28.256110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:28.256110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:28.270792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:28.392743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:28.395294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62b580-c092-4a34-8e6a-ca08ca535938 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:28.411498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1121/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:28.936091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:29.010184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:29.031263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44b36f-eb67-4053-bc5d-407d36c68bd1 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:29.046491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:29.047414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:29.047862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:29.048667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] PATH_INFO: `/v3/users/0b83183a946e4d22b077eb03e41428b5` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:29.050321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b83183a946e4d22b077eb03e41428b5)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:29.054012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:29.201014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b83183a946e4d22b077eb03e41428b5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:29.206243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b83183a946e4d22b077eb03e41428b5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:29.217624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b83183a946e4d22b077eb03e41428b5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:29.231742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b83183a946e4d22b077eb03e41428b5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:29.272576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b83183a946e4d22b077eb03e41428b5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:29.310311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1124/2245] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:28 2022] DELETE /identity/v3/users/0b83183a946e4d22b077eb03e41428b5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:29.323165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511713-3404-4222-90b9-00e7fb721999 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:29.326362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511713-3404-4222-90b9-00e7fb721999 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:29.327391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511713-3404-4222-90b9-00e7fb721999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:29.421866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-77511713-3404-4222-90b9-00e7fb721999 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:29.429369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-77511713-3404-4222-90b9-00e7fb721999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:29.638388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1122/2246] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:29.655539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:29.658990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:29.803475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:29.803927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:29.804385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:29.845170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ce3b7-c615-4ea2-acca-3a0ad87c0562 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:29.850624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:29.851445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:29.851828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:29.852098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:29.853481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:29.857130 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:29.864558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:29.866195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:29.878769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1125/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:29.993185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1123/2248] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:55:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:30.017666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:30.021315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:30.235651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a48d13-fa73-40fe-8655-512401e4989d service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:30.244192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:30.245225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:30.245676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:30.246122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:30.261039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:30.263659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:30.273833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1126/2249] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:55:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:55:30.339192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:30.342689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:30.388080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d3f2-affd-4602-b5ca-ab85b9667437 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:30.395580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:30.396273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:30.396551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:30.397315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] PATH_INFO: `/v3/projects/3b7883c9328040839a7dd086474442fd` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:30.399140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7883c9328040839a7dd086474442fd)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:30.401574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:30.532451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:30.543881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7883c9328040839a7dd086474442fd', 'request_id': 'req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:30.567854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7883c9328040839a7dd086474442fd', 'request_id': 'req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:30.597172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7883c9328040839a7dd086474442fd', 'request_id': 'req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:30.625459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] The token cache is being invalidate because project 3b7883c9328040839a7dd086474442fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:30.627720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:30.632220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:30.639024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1124/2250] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:30 2022] DELETE /identity/v3/projects/3b7883c9328040839a7dd086474442fd => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:30.645970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:30.654076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:30.679762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6273b-766f-4283-a807-9076cb3024e2 service swift] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:31.010830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:31.012209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:31.013030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:31.013854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] PATH_INFO: `/v3/users/cb010201ec494d69b31137624da221fc` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:31.016664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb010201ec494d69b31137624da221fc)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:31.021680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:31.227321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb010201ec494d69b31137624da221fc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:31.229146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb010201ec494d69b31137624da221fc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:31.247737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb010201ec494d69b31137624da221fc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:31.266746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb010201ec494d69b31137624da221fc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:31.308039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-e2c2ba7a-9631-4269-897a-a1c93b2e34cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb010201ec494d69b31137624da221fc'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:31.341819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1127/2251] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:30 2022] DELETE /identity/v3/users/cb010201ec494d69b31137624da221fc => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:32.688667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:32.691451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:32.710208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34727c91-8d5c-4e94-8df8-a52a2f3cb23f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:32.809630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:32.810306 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:32.811273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:32.811613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] PATH_INFO: `/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:32.812923 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93aaafcc874c4ce7b5f1a1645df645fe)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:32.815080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:32.886433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93aaafcc874c4ce7b5f1a1645df645fe', 'request_id': 'req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:32.895764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93aaafcc874c4ce7b5f1a1645df645fe', 'request_id': 'req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:32.907872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93aaafcc874c4ce7b5f1a1645df645fe', 'request_id': 'req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:32.925560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] The token cache is being invalidate because project 93aaafcc874c4ce7b5f1a1645df645fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:32.926114 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:32.927268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a6f9a18c-48b1-40aa-857c-f0f440fd2b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:32.933198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1125/2252] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:32 2022] DELETE /identity/v3/projects/93aaafcc874c4ce7b5f1a1645df645fe => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:32.935466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:32.936234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:32.936234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:32.960621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:32.962586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:33.135536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1128/2253] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:33.144316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.144710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.145094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.170825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:33.172351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:33.182281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1126/2254] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:33.187395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.204722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f7d1b-77bc-485d-8feb-c8c17019e606 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:33.221544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:33.222664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.223146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.225120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.226580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:33.227145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:33.230920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:33.254769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.266115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1129/2255] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:55:33.267724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76a8c6-7a6a-484a-96c4-1d1a8617892a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:33.274539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:33.274539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.275044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.275373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.276191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:33.278756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:33.323531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.332556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1127/2256] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:33 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:33.345135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae10b-aada-44b0-9a23-6fbb0371a0a8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:33.358757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:33.543086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:33.543900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.544284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.544656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.546318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:33.551604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f64ea-8914-47c1-9c3b-2d743ce94ca4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:33.612175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.636900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1130/2257] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:33 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:33.654745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6dcf-84ea-4653-808c-a81b2066d01b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:33.678522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:33.679676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.681468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.685233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.687561 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7cd0-7769-410e-ab14-217f02c8ad7b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.687820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7cd0-7769-410e-ab14-217f02c8ad7b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.688198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7cd0-7769-410e-ab14-217f02c8ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.699684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:33.710886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:33.756038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1128/2258] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:33.757250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.757841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-cddd7cd0-7769-410e-ab14-217f02c8ad7b None None] MFA Rules not processed for user `a8cbfc74644f4d4594a70b833ad7002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:33.761153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cddd7cd0-7769-410e-ab14-217f02c8ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:33.796140 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b91982-6ef1-4a3c-8d27-ef0c9b205a34 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:33.815667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:33.823772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:33.827511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:33.827511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:33.831328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:33.839816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:33.885073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1129/2259] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:33.900188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:33.982126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71618cd1-82d4-4988-b789-8ae1d762ecce admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:34.013211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:34.015101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:34.015708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:34.016214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] PATH_INFO: `/v3/projects/b6349d41148942bf8bcce7a7985ea29b/users/8a84e2429d0e4b38add3d5543f5730bb/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:34.032000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:34.047031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6349d41148942bf8bcce7a7985ea29b, user_id=8a84e2429d0e4b38add3d5543f5730bb, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:34.064141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:34.106729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1131/2260] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:34.114107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1130/2261] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:33 2022] PUT /identity/v3/projects/b6349d41148942bf8bcce7a7985ea29b/users/8a84e2429d0e4b38add3d5543f5730bb/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:34.147179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:34.153984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:34.154446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:34.241575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] MFA Rules not processed for user `8a84e2429d0e4b38add3d5543f5730bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:34.242960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:34.429066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1132/2262] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:34.447105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:34.449875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:34.587643 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671459-298c-439f-ac0a-97b935abdad9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:34.594384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:34.595518 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:34.596061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:34.596536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:34.608278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:34.610198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:34.624120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1131/2263] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:34.663629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:34.782761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:34.860999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8c032-e621-41df-9b58-bfab52e01617 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:34.873454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:34.874102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:34.874652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:34.875882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] PATH_INFO: `/v3/users/8a84e2429d0e4b38add3d5543f5730bb` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:34.877937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a84e2429d0e4b38add3d5543f5730bb)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:34.881879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:35.032168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a84e2429d0e4b38add3d5543f5730bb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.034888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a84e2429d0e4b38add3d5543f5730bb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.074107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a84e2429d0e4b38add3d5543f5730bb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.094546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:35.103024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:35.103326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a84e2429d0e4b38add3d5543f5730bb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.172629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7fa40212-b505-4144-b013-4a5beb3bdfa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a84e2429d0e4b38add3d5543f5730bb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.198635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9411-5158-4146-8d90-92fbb33463a5 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:35.201120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1133/2264] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:34 2022] DELETE /identity/v3/users/8a84e2429d0e4b38add3d5543f5730bb => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:35.209771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:35.210405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:35.211112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:35.266130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:35.270764 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:35.413095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1134/2265] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:35.419490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:35.420385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:35.420837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:35.421254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] PATH_INFO: `/v3/users/13b8d70f2288460e8ee4436379c619d7` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:35.423982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b8d70f2288460e8ee4436379c619d7)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:35.426878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:35.431196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:35.444727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:35.629150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b8d70f2288460e8ee4436379c619d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.629150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b8d70f2288460e8ee4436379c619d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.629150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b8d70f2288460e8ee4436379c619d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.652325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b8d70f2288460e8ee4436379c619d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.658254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d673d-e8bd-48d3-bd33-d48b5ecb64d9 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:35.671905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:35.674228 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:35.676105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:35.676643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:35.681399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:35.700989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5b87227e-e42b-47ec-aaea-2df27a03add5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b8d70f2288460e8ee4436379c619d7'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:35.723305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:35.745069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:35.745856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:35.756450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1132/2266] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:35 2022] DELETE /identity/v3/users/13b8d70f2288460e8ee4436379c619d7 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:35.763397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:35.763397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:35.763397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:35.766167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1135/2267] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:35.855670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:35.856253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:35.874370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:36.033596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1133/2268] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:36.133313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:36.136212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:36.179393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696abb7-2a0a-43cc-adcc-a5932f9402bd service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:36.321137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:36.322149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:36.322535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:36.322800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:36.339405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:36.341503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7cfe9-aedd-488e-a1c6-74768ec6a14f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:36.359674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1136/2269] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:36.622397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:36.625074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:36.686650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c8b20-b642-41cb-a5af-ab33b55f1ab6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:36.717850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:36.723473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:36.725042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:36.727451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] PATH_INFO: `/v3/projects/b6349d41148942bf8bcce7a7985ea29b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:36.732095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6349d41148942bf8bcce7a7985ea29b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:36.739459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:36.894927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6349d41148942bf8bcce7a7985ea29b', 'request_id': 'req-8026b187-d03d-44a4-93f4-240cb56b71e8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:36.911906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6349d41148942bf8bcce7a7985ea29b', 'request_id': 'req-8026b187-d03d-44a4-93f4-240cb56b71e8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:36.930882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6349d41148942bf8bcce7a7985ea29b', 'request_id': 'req-8026b187-d03d-44a4-93f4-240cb56b71e8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:36.956608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] The token cache is being invalidate because project b6349d41148942bf8bcce7a7985ea29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:36.957945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:36.960155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8026b187-d03d-44a4-93f4-240cb56b71e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:36.969428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1134/2270] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:36 2022] DELETE /identity/v3/projects/b6349d41148942bf8bcce7a7985ea29b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:36.972474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:36.972929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:36.973225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:37.100267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:37.102220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:37.307424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1137/2271] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:37.378688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:37.379120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:37.379515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:37.419853 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:37.421616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:37.437097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1135/2272] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:37.439036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:37.454091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b6f77-b1ec-46a5-856e-87ffbd84bc10 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:37.464186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:37.464935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:37.465486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:37.465849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:37.471932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:55:37.472275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:37.474131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:37.549627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1138/2273] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:55:37.554546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:37.575537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18439fb-b4ea-4fc0-8ad6-73ef80b414d8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:37.592624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:37.595249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:37.595636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:37.595891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:37.599608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:37.603044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:37.681779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1136/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:37 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:37.697017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:37.730664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18179cb7-cee6-4539-889f-b2dc16ad809b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:37.947938 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:37.950854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:37.951368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:37.952610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:37.954644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:37.960192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:38.046810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1139/2275] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:37 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:38.049231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:38.099696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2be8b-0282-4bb2-a2c0-55602bb4285f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:38.139813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:38.140782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:38.141182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:38.141507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:38.142348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:38.144619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:38.165887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1137/2276] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:38.169604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:38.196405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532ffc8-80af-4b71-b638-4aaa15f3b6df admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:38.214809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:38.215672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:38.215979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:38.216229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:38.216859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:38.218507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412d37-adc2-4f4c-b57b-fd595b6ae173 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:38.258933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1140/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:38.261218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:38.296898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3891fb-e7b8-4b21-a385-4911632aaa9f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:38.316903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:38.319962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:38.321151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:38.322416 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] PATH_INFO: `/v3/projects/5226187c5c2d4b9694b5e5f9c23aa843/users/e0f1542b38514de1b0e4dc57a240aceb/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:38.394176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5226187c5c2d4b9694b5e5f9c23aa843, user_id=e0f1542b38514de1b0e4dc57a240aceb, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:38.404646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:38.489567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1138/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:38 2022] PUT /identity/v3/projects/5226187c5c2d4b9694b5e5f9c23aa843/users/e0f1542b38514de1b0e4dc57a240aceb/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:55:38.518252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:38.519610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:38.520063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:38.569415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:38.646080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] MFA Rules not processed for user `e0f1542b38514de1b0e4dc57a240aceb`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:38.650033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:38.886312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1141/2279] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:38.891952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:38.987576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:38.994556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:39.088847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:39.181845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c04487-d584-4028-92bc-e7a60c1cfce6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.196493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.197147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.197427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.197707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.198365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.200470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.243350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978d5e7-b608-4216-a403-4561baacc83c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.253027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.255182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.256088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.256923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] PATH_INFO: `/v3/projects/008bda799c4a4110ba61171e4f2f9890` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.260323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008bda799c4a4110ba61171e4f2f9890)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.265255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.265582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1139/2280] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:55:38 2022] POST /identity/v3/projects => generated 373 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:55:39.271301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:39.417757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008bda799c4a4110ba61171e4f2f9890', 'request_id': 'req-190100a2-aa5b-4a09-aced-7f556fc33ab7'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:39.442219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c8373-1526-4492-afa0-80f7edacdfb6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.444542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008bda799c4a4110ba61171e4f2f9890', 'request_id': 'req-190100a2-aa5b-4a09-aced-7f556fc33ab7'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:39.453550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008bda799c4a4110ba61171e4f2f9890', 'request_id': 'req-190100a2-aa5b-4a09-aced-7f556fc33ab7'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:39.472579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] The token cache is being invalidate because project 008bda799c4a4110ba61171e4f2f9890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:39.472907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:39.473669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:39.479875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1142/2281] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:38 2022] DELETE /identity/v3/projects/008bda799c4a4110ba61171e4f2f9890 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:39.590750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.591731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.592022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.592295 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.592988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.595110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.630834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1140/2282] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:55:39 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:55:39.639269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:39.639269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:39.704681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190100a2-aa5b-4a09-aced-7f556fc33ab7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.713407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.714426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.714901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.715356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.716377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.719467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.734587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1143/2283] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:55:39.739928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:39.751200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a958fb-660b-4658-abb7-021297674da6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.756124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.756807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.757082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.757376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] PATH_INFO: `/v3/projects/cdf66f2e730d46acbe68851e272c1b63/users/2877bb4b3626415a8e5875fc182e3539/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.781628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf66f2e730d46acbe68851e272c1b63, user_id=2877bb4b3626415a8e5875fc182e3539, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.785129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.808397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1141/2284] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:39 2022] PUT /identity/v3/projects/cdf66f2e730d46acbe68851e272c1b63/users/2877bb4b3626415a8e5875fc182e3539/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:39.815606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:39.826873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca3940-893f-4d03-9e85-5e6011f23e10 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.840722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:39.940499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.941142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.941423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.941696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.943039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:39.945992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5ba05-d1f4-47ce-ab7b-6cc28806d53e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:39.960814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1144/2285] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:55:39.967441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:39.988321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25c48b-15b1-4a4f-8970-4a5df78c0e44 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:39.994053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:39.994949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:39.995426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:39.995847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] PATH_INFO: `/v3/projects/cdf66f2e730d46acbe68851e272c1b63/users/2877bb4b3626415a8e5875fc182e3539/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:39.999073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf66f2e730d46acbe68851e272c1b63, user_id=2877bb4b3626415a8e5875fc182e3539, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:40.002594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:40.024017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1142/2286] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:55:39 2022] PUT /identity/v3/projects/cdf66f2e730d46acbe68851e272c1b63/users/2877bb4b3626415a8e5875fc182e3539/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:40.050902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1fa8-2fa9-4efc-9ba5-ddf675d855a8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:40.051437 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1fa8-2fa9-4efc-9ba5-ddf675d855a8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:40.051861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af1fa8-2fa9-4efc-9ba5-ddf675d855a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:40.093314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d2af1fa8-2fa9-4efc-9ba5-ddf675d855a8 None None] MFA Rules not processed for user `2877bb4b3626415a8e5875fc182e3539`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:40.094565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d2af1fa8-2fa9-4efc-9ba5-ddf675d855a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:40.278332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1145/2287] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:55:40.292478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:40.294566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:40.324798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:40.380483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b862-6a03-41fa-bafc-77e43a29516f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:40.391934 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:40.392762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:40.393040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:40.393585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:40.395883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:40.532904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:40.534386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:40.545951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1143/2288] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:55:43.350538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:43.351272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:43.351272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:43.372286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:43.373627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:43.476910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1146/2289] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:55:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:43.490938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:43.491609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:43.493509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:43.570862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:43.613392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064438b1-c31e-4b4f-ab74-b92fcb4632b8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:43.620646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:43.621560 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:43.622082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:43.622544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] PATH_INFO: `/v3/users/e0f1542b38514de1b0e4dc57a240aceb` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:43.623937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f1542b38514de1b0e4dc57a240aceb)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:43.626115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:43.627760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13a97-971a-4d71-b967-36cc4c7a6c16 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:43.635627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:43.636384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:43.636731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:43.637065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:43.646952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:43.649274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8dbfa-07bd-4777-b862-b8460b57d88b service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:43.659121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1144/2290] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:55:43.703056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f1542b38514de1b0e4dc57a240aceb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:43.704634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f1542b38514de1b0e4dc57a240aceb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:43.716861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f1542b38514de1b0e4dc57a240aceb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:43.727241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f1542b38514de1b0e4dc57a240aceb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:43.762278 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f1542b38514de1b0e4dc57a240aceb'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:43.774017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1147/2291] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:43 2022] DELETE /identity/v3/users/e0f1542b38514de1b0e4dc57a240aceb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:43.780728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:43.781834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:43.787887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:43.829261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:43.833741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:43.945248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:43.979367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1145/2292] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:43.993771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:43.999571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:44.252820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b687e-c302-4800-a76c-a74a9a316412 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:44.260748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:44.261402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:44.262050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:44.262330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:44.289781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:44.291894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:44.308221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1148/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:55:44.918596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:44.920420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:44.966908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718545-0f24-4591-bbd8-40d509d1f40c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:44.976676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:44.977656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:44.977944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:44.978378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] PATH_INFO: `/v3/projects/5226187c5c2d4b9694b5e5f9c23aa843` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:44.979799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5226187c5c2d4b9694b5e5f9c23aa843)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:44.982852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:45.091424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5226187c5c2d4b9694b5e5f9c23aa843', 'request_id': 'req-36a60304-d679-4d22-9cf2-7c317bd8a101'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.101707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5226187c5c2d4b9694b5e5f9c23aa843', 'request_id': 'req-36a60304-d679-4d22-9cf2-7c317bd8a101'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.114618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5226187c5c2d4b9694b5e5f9c23aa843', 'request_id': 'req-36a60304-d679-4d22-9cf2-7c317bd8a101'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.130598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] The token cache is being invalidate because project 5226187c5c2d4b9694b5e5f9c23aa843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:45.131090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.132090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.135504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1146/2294] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:44 2022] DELETE /identity/v3/projects/5226187c5c2d4b9694b5e5f9c23aa843 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:45.141920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:45.156583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0717-ed52-4eb4-a4fe-e6b622121eaa service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:45.250769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:45.251517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:45.251797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:45.252068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] PATH_INFO: `/v3/users/2877bb4b3626415a8e5875fc182e3539` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:45.254484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2877bb4b3626415a8e5875fc182e3539)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:45.256519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:45.264895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:55:45.353786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2877bb4b3626415a8e5875fc182e3539'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.354777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2877bb4b3626415a8e5875fc182e3539'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.376760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2877bb4b3626415a8e5875fc182e3539'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.395010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2877bb4b3626415a8e5875fc182e3539'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.440756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d4a9313c-a055-4b12-8a05-2c60f31564ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2877bb4b3626415a8e5875fc182e3539'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:45.447968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1149/2295] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:55:45 2022] DELETE /identity/v3/users/2877bb4b3626415a8e5875fc182e3539 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:46.265367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:55:46.267116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:46.295109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60304-d679-4d22-9cf2-7c317bd8a101 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:55:46.413788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:55:46.416195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:46.417944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:46.418727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] PATH_INFO: `/v3/projects/cdf66f2e730d46acbe68851e272c1b63` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:46.421651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf66f2e730d46acbe68851e272c1b63)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:55:46.426615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:55:46.507989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf66f2e730d46acbe68851e272c1b63', 'request_id': 'req-167fc0f7-1300-4814-ad4a-d88048fc90ca'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:46.517998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf66f2e730d46acbe68851e272c1b63', 'request_id': 'req-167fc0f7-1300-4814-ad4a-d88048fc90ca'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:46.532379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf66f2e730d46acbe68851e272c1b63', 'request_id': 'req-167fc0f7-1300-4814-ad4a-d88048fc90ca'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:46.550714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] The token cache is being invalidate because project cdf66f2e730d46acbe68851e272c1b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:55:46.554789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:46.559434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-167fc0f7-1300-4814-ad4a-d88048fc90ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:55:46.562309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1147/2296] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:55:46 2022] DELETE /identity/v3/projects/cdf66f2e730d46acbe68851e272c1b63 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:55:49.913744 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f60c8e-9e2f-41da-997e-ae00aed5d793 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:55:49.914225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f60c8e-9e2f-41da-997e-ae00aed5d793 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:55:49.914225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f60c8e-9e2f-41da-997e-ae00aed5d793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:55:49.951918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e3f60c8e-9e2f-41da-997e-ae00aed5d793 None None] MFA Rules not processed for user `76eb39c2010e43aaa0a01caaea3a38b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:55:49.953186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e3f60c8e-9e2f-41da-997e-ae00aed5d793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:55:50.094025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1150/2297] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:55:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:55:56.545181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=78,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:00.084835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:02.735119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:02.735882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:02.735882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:02.737385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1148/2298] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:56:02.744665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80546310-3d3a-404c-803c-5e5b374d25c4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:02.745297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80546310-3d3a-404c-803c-5e5b374d25c4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:02.745297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80546310-3d3a-404c-803c-5e5b374d25c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:02.771934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-80546310-3d3a-404c-803c-5e5b374d25c4 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:56:02.773153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-80546310-3d3a-404c-803c-5e5b374d25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:02.870192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1151/2299] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:56:02.889177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:02.891248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:02.984701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455e424-83a6-407f-ac89-698ca4e9a08a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:02.991382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:02.992441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:02.992925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:02.993397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.009321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:03.012404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4be17c-930f-4962-b424-521aa3c90162 service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:03.027693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1149/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:56:03.181257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.181637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.182378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.186029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1152/2301] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:56:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:56:03.193301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6e4e-50b3-4423-b858-bbb617e76bcd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.193771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6e4e-50b3-4423-b858-bbb617e76bcd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.194195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a6e4e-50b3-4423-b858-bbb617e76bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.230801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-771a6e4e-50b3-4423-b858-bbb617e76bcd None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:56:03.236525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-771a6e4e-50b3-4423-b858-bbb617e76bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:03.322997 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1150/2302] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:56:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:56:03.342445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:03.345354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:03.361023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52932-88d0-498b-95dc-d2aa20056feb None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:03.366624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:03.368968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.368968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.368968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.378340 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:03.380159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520b3b8-82cb-45da-b3a5-ca27efacd2eb service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:03.391800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1153/2303] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:56:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:56:03.545771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb0b9-c904-492f-b9a5-f0353b243a6e None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.546012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb0b9-c904-492f-b9a5-f0353b243a6e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.546300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb0b9-c904-492f-b9a5-f0353b243a6e None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.547792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1151/2304] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:56:03.554083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.554353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.554620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.576839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:56:03.577979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:03.588613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1154/2305] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:56:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:56:03.600491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb0b9-c904-492f-b9a5-f0353b243a6e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:03.613575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb0b9-c904-492f-b9a5-f0353b243a6e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:03.617432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:03.618011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:03.618288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:03.618680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:03.629799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:03.631284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:03.644062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1152/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:56:05.278510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:05.280034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:05.424232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922b9d2-f93f-4d2a-97fd-0787bd61d600 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:05.429731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:05.430367 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:05.430647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:05.430924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:05.432883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:05.454203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:05.455878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9574f-8cdb-4d2e-b4cf-b9b98a0e7da4 service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:05.468120 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1155/2307] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:56:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:56:10.057974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:10.060427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:10.080947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d48f-0a73-4b2b-9454-46cdc4cd56a4 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:10.084660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:10.085267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:10.085535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:10.085824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:10.087812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:10.163812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:10.165268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:10.175397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1153/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:56:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:56:13.495052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826680-f26f-4222-9d1a-1d683754b40d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:13.495794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826680-f26f-4222-9d1a-1d683754b40d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:13.495963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826680-f26f-4222-9d1a-1d683754b40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:13.522936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-09826680-f26f-4222-9d1a-1d683754b40d None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:56:13.526630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-09826680-f26f-4222-9d1a-1d683754b40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:13.604332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1156/2309] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:56:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:56:13.620431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:13.622784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:13.681976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:13.738957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d54db-4e24-4f6f-b132-ef4f504e6b74 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:13.748460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:13.748754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:13.749264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:13.749799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:13.764930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:13.767389 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:13.777325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1154/2310] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:56:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:56:23.596804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:23.773870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=17 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:39.949944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b469a8-1830-4d27-8e71-02289f046a6f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:39.950558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b469a8-1830-4d27-8e71-02289f046a6f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:39.950558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b469a8-1830-4d27-8e71-02289f046a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:39.973863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d7b469a8-1830-4d27-8e71-02289f046a6f None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:56:39.975791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d7b469a8-1830-4d27-8e71-02289f046a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:56:39.985670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1157/2311] 10.222.0.26 () {58 vars in 967 bytes} [Mon Sep 12 23:56:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:56:40.000686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:56:40.014195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500b0c7-2374-42ba-a72c-90ce0aecd907 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:56:40.020156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:56:40.020790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:56:40.021186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:56:40.021405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:56:40.032714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:56:40.034338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a99284-e491-4cd2-b291-088ba361d12f service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:56:40.043649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1155/2312] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:56:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:56:49.966754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:56:50.039772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:57:07.479309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fa46e-548b-492c-b1c0-250a91ff7d1f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:07.480255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fa46e-548b-492c-b1c0-250a91ff7d1f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:07.480613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fa46e-548b-492c-b1c0-250a91ff7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:07.501355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c04fa46e-548b-492c-b1c0-250a91ff7d1f None None] MFA Rules not processed for user `a8cbfc74644f4d4594a70b833ad7002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:57:07.502565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c04fa46e-548b-492c-b1c0-250a91ff7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:57:07.580130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1158/2313] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:57:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:57:17.573181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=25 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:57:30.195734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e123926-51b3-4c87-897c-6658703fec3c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:30.196385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e123926-51b3-4c87-897c-6658703fec3c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:30.196385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e123926-51b3-4c87-897c-6658703fec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:30.216081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3e123926-51b3-4c87-897c-6658703fec3c None None] MFA Rules not processed for user `76eb39c2010e43aaa0a01caaea3a38b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:57:30.217307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3e123926-51b3-4c87-897c-6658703fec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:57:30.224478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1156/2314] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:57:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:57:33.241132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:33.241610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:33.241721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:33.263420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] MFA Rules not processed for user `296f02c73fc548969bf60df4f8f372b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:57:33.264653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:57:33.333975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1159/2315] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:57:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:57:40.210979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:57:43.323330 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=25 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:57:56.228533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:56.229285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:56.229285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:56.251545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:57:56.252810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:57:56.261276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1157/2316] 10.222.0.26 () {58 vars in 967 bytes} [Mon Sep 12 23:57:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:57:56.314917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:57:56.329825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5425b-843f-4465-abe4-017a235cf4ed None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:57:56.336359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:57:56.337001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:56.337385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:56.337545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:56.347705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:57:56.349343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb3b76-ff90-460a-97f6-8e1cea83cedb service swift] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:57:56.358723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1160/2317] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:57:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:57:58.838915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:57:58.855442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096811b6-cee2-4f84-bc23-ff41233fd27f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:57:58.862167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:57:58.864070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:57:58.865091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:57:58.865884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:57:58.869982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:57:58.901809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:57:58.903360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8aac-c6aa-4925-a9d1-9ff8c94ec0c6 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:57:58.917849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1158/2318] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:57:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:00.083210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:00.084274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:00.084785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:00.086451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1161/2319] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:58:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:58:00.094919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:00.095542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:00.095542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:00.120355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] MFA Rules not processed for user `734ed3e452894cbbbe292b78390dc2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:00.121950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:00.133061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1159/2320] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:58:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:00.148948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:00.150892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:00.169527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c10ca-6d65-46b5-8109-4b910a2ae5a8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:00.175127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:00.175887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:00.176239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:00.176626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:00.180086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:00.187870 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:00.189591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:00.194311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8887f-10f2-4801-8225-3aa663326876 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:00.202378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:00.202789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:00.203242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:00.203693 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:00.205920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1162/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:00.206952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:00.232677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:00.236239 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:00.259608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1160/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:58:04.387247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:04.390719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:04.460522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753f87-c533-4ad5-9340-449aeab3c29b service placement] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:04.465336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:04.465978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:04.466253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:04.466520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] PATH_INFO: `/v3/users/e76b3353ce4b4361872ed6dba9f895d6` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:04.467690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76b3353ce4b4361872ed6dba9f895d6)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:04.469666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:04.519380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76b3353ce4b4361872ed6dba9f895d6'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.520023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76b3353ce4b4361872ed6dba9f895d6'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.527639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76b3353ce4b4361872ed6dba9f895d6'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.536769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76b3353ce4b4361872ed6dba9f895d6'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.549785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76b3353ce4b4361872ed6dba9f895d6'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.555940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1163/2323] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:58:04 2022] DELETE /identity/v3/users/e76b3353ce4b4361872ed6dba9f895d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:04.740929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:04.755344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323636d-acd9-4ea4-bf46-3e12ce0488b2 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:04.828361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:04.829009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:04.829284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:04.829558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] PATH_INFO: `/v3/projects/880f7d1c88224ebd85d15f7c43e7fe72` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:04.830730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880f7d1c88224ebd85d15f7c43e7fe72)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:04.832706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:04.890731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880f7d1c88224ebd85d15f7c43e7fe72', 'request_id': 'req-98d15ed7-7303-4553-a09a-ae58d79d46b9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.899188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880f7d1c88224ebd85d15f7c43e7fe72', 'request_id': 'req-98d15ed7-7303-4553-a09a-ae58d79d46b9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.909624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880f7d1c88224ebd85d15f7c43e7fe72', 'request_id': 'req-98d15ed7-7303-4553-a09a-ae58d79d46b9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.922871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] The token cache is being invalidate because project 880f7d1c88224ebd85d15f7c43e7fe72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:58:04.923253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.924304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:04.926716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1161/2324] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:58:04 2022] DELETE /identity/v3/projects/880f7d1c88224ebd85d15f7c43e7fe72 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:04.931007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:04.932735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:04.947926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f751d5-08b9-4ad8-99af-36619a5ac147 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:05.002702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:05.016230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:05.019505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:05.019505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:05.019505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] PATH_INFO: `/v3/users/3ec6216ba1c64fdca97e3705b683f999` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:05.020476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec6216ba1c64fdca97e3705b683f999)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:05.022785 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:05.080116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec6216ba1c64fdca97e3705b683f999'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.081202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec6216ba1c64fdca97e3705b683f999'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.094980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec6216ba1c64fdca97e3705b683f999'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.110617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec6216ba1c64fdca97e3705b683f999'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.132099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec6216ba1c64fdca97e3705b683f999'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.138584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1164/2325] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:58:04 2022] DELETE /identity/v3/users/3ec6216ba1c64fdca97e3705b683f999 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:58:05.340041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:05.341573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:05.364797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15ed7-7303-4553-a09a-ae58d79d46b9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:05.418089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:05.432630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:05.433323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:05.433600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:05.433870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] PATH_INFO: `/v3/projects/b1da04c606664831adb0d1b12d7815b2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:05.435004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1da04c606664831adb0d1b12d7815b2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:05.437074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:05.493169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1da04c606664831adb0d1b12d7815b2', 'request_id': 'req-b95111cd-0570-4e08-bba7-bd47e5dc4551'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.501885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1da04c606664831adb0d1b12d7815b2', 'request_id': 'req-b95111cd-0570-4e08-bba7-bd47e5dc4551'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.513339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1da04c606664831adb0d1b12d7815b2', 'request_id': 'req-b95111cd-0570-4e08-bba7-bd47e5dc4551'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.524532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] The token cache is being invalidate because project b1da04c606664831adb0d1b12d7815b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:58:05.524816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.525454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:05.526968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1162/2326] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:58:05 2022] DELETE /identity/v3/projects/b1da04c606664831adb0d1b12d7815b2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:58:15.134834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:15.521190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:19.324765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:19.326432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:19.394092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82699fbf-0a71-4358-bf65-17c839a1e6e5 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:19.453462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:19.454125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:19.454405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:19.454675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:19.456670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:19.524935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:19.526397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:19.537542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1165/2327] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:27.049020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:27.050512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:27.119179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95111cd-0570-4e08-bba7-bd47e5dc4551 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:27.123679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:27.124567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:27.125074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:27.125565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:27.128072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:27.226743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:27.228937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eac46f-f2b4-4940-afcc-f92e3535826b service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:27.244978 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1163/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:58:29.531997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=68 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:31.504978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:31.507565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:31.635793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee2c8c-ff84-4d96-9e0a-4b15f94e29e0 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:31.646020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:31.647877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:31.648387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:31.648876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:31.656811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:31.745658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:31.752803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:31.775586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1166/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:33.161861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:33.162489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:33.162489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:33.191342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] MFA Rules not processed for user `a8cbfc74644f4d4594a70b833ad7002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:33.193415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:33.206871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1164/2330] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:33.676200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:33.688301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dcaff-315b-4832-b1dc-c330b83b6b48 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:33.692226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:33.692859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:33.693138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:33.693410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:33.695290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:33.706675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:33.708202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:33.718211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1167/2331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:34.258719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:34.260616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:34.365159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd270-7d71-4147-aaef-e6b9b05aab98 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:34.375897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:34.377148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:34.377646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:34.378006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:34.381017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:34.394696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:34.396984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:34.411818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1165/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:43.713650 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=62 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:44.404177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=83 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:45.443499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:45.445875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:45.460462 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ed940-ec40-4cb4-b863-84f4d1048669 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:45.464777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:45.465401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:45.465675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:45.465947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:45.467882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:45.569978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:45.572822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:45.589536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1168/2333] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:46.156213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:46.172487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1d175-191c-4a1b-9f59-cd598b58dca5 service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:46.179538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:46.180566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:46.181050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:46.181515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:46.184575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:46.201951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:46.204622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:46.221370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1166/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:50.416927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:50.418763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:50.433652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad160da-d7f0-4bf5-86cd-f05bf154d643 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:50.439705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:50.440566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:50.440917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:50.441247 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] PATH_INFO: `/v3/users/76eb39c2010e43aaa0a01caaea3a38b9` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:50.442882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76eb39c2010e43aaa0a01caaea3a38b9)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:50.444963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:50.519681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76eb39c2010e43aaa0a01caaea3a38b9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.521002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76eb39c2010e43aaa0a01caaea3a38b9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.534132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76eb39c2010e43aaa0a01caaea3a38b9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.545715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76eb39c2010e43aaa0a01caaea3a38b9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.565921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80f58d0f-1d78-48e1-a09b-458e9b8582f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76eb39c2010e43aaa0a01caaea3a38b9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.573799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1169/2335] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:58:50 2022] DELETE /identity/v3/users/76eb39c2010e43aaa0a01caaea3a38b9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:50.805759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:50.825162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec75a-d79e-42f3-a5c5-f4ede7b0bd81 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:50.915177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:50.917727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:50.917727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:50.917727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] PATH_INFO: `/v3/projects/6c458cf41ded4113a09fb07e10d81d39` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:50.918216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c458cf41ded4113a09fb07e10d81d39)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:50.920034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:50.985776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c458cf41ded4113a09fb07e10d81d39', 'request_id': 'req-4d3b3979-365b-4802-a360-4ea2040b42c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:50.993500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c458cf41ded4113a09fb07e10d81d39', 'request_id': 'req-4d3b3979-365b-4802-a360-4ea2040b42c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:51.010891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c458cf41ded4113a09fb07e10d81d39', 'request_id': 'req-4d3b3979-365b-4802-a360-4ea2040b42c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:51.031885 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] The token cache is being invalidate because project 6c458cf41ded4113a09fb07e10d81d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:58:51.032980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:51.033795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4d3b3979-365b-4802-a360-4ea2040b42c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:51.036737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1167/2336] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:58:50 2022] DELETE /identity/v3/projects/6c458cf41ded4113a09fb07e10d81d39 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:51.041989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.042903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.043458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.076802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:51.077965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:51.171457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1170/2337] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:58:51.173067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.173359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.173714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.196145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:51.197396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:51.205833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1168/2338] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:51.210651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.221854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae752b2a-2bc0-4ee6-84f5-73ed62fe8ba6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.226961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.227657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.227934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.228205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.228917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:58:51.229221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.230947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.233132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:51.248828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1171/2339] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:58:51.253990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.265007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8ae03-1809-49b0-84af-ec62bf51b172 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.270103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.271354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.271825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.272142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.273025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.275198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.307763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1169/2340] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:51.312713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.323921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc76a7-745d-43d2-9f1a-cb68be5af45b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.392078 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.392885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.393312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.393725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.394525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.397436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.427081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1172/2341] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:51.429095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.445617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d85d9-57ee-4d60-a0ec-7e1bfcd1530a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.451372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.452047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.452350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.452591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.453281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.455305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.470887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1170/2342] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:51.475941 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.489538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bc0c3-10f0-4aee-8f7a-a47b5c757036 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.494676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.496007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.496301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.496567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.497425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.499156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.516505 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1173/2343] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:51.518025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.524400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.526138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:51.530318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f67da-b65a-4e06-9cb0-ff6688553b1a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.536525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.537369 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.537718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.538067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] PATH_INFO: `/v3/projects/3d09f7144d554e2d8a1c2e0177f3a81b/users/8ef54e01a8a6413e809abe8f9427da14/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.556998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d09f7144d554e2d8a1c2e0177f3a81b, user_id=8ef54e01a8a6413e809abe8f9427da14, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.560634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d871da3-4931-4a88-a2d1-07c49ffaa506 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.582418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1171/2344] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:51 2022] PUT /identity/v3/projects/3d09f7144d554e2d8a1c2e0177f3a81b/users/8ef54e01a8a6413e809abe8f9427da14/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:51.606695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.607302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.607995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.654623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:51.659267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] MFA Rules not processed for user `8ef54e01a8a6413e809abe8f9427da14`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:51.661489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:51.664907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928d835-f794-4e16-82c4-bd92bda1858e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.773903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.775400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.775825 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.776283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.778686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:51.791290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1172/2345] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:51.794898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:51.795958 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.799370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.801755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.824141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1174/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:58:51.918825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee219-4bf3-42b4-a273-af053ad6b775 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:51.925816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:51.926467 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:51.926747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:51.927014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:51.927701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:51.930227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:51.967261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1173/2347] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:58:51.971047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:51.982844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829593b-9ba0-4ada-88b9-c7a24d9897b5 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.051952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.052602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.052881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.053145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.054037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.055867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.091854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1175/2348] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:58:51 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:58:52.095519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:52.113231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda81d-6f38-41c0-abac-57d0ea00fbc2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.121063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.122112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.122569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.123051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.124394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.127732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.141106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1174/2349] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:58:52.145928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:52.157388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d869-2aed-43fc-939d-a1f0e726f0b1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.163539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.164125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.164389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.164695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] PATH_INFO: `/v3/projects/42f936d2a52945ffb0fb540539d36e36/users/641543d9429a405c8e435ca344ea5fe3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.183566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f936d2a52945ffb0fb540539d36e36, user_id=641543d9429a405c8e435ca344ea5fe3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.186617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.214792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1176/2350] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:52 2022] PUT /identity/v3/projects/42f936d2a52945ffb0fb540539d36e36/users/641543d9429a405c8e435ca344ea5fe3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:52.219500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:52.237425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1795142-48a5-4341-93ec-664aa6552d45 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.363823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.364879 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.365382 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.365886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.367012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.371072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90774f4c-0005-4508-a4fa-8c1854f90433 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.398764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1175/2351] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:58:52.400268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:52.423214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b442-2686-4906-9293-8b831d7dd108 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.434312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.438226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.440253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.441149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] PATH_INFO: `/v3/projects/42f936d2a52945ffb0fb540539d36e36/users/641543d9429a405c8e435ca344ea5fe3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.445881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f936d2a52945ffb0fb540539d36e36, user_id=641543d9429a405c8e435ca344ea5fe3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.452693 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.476103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1177/2352] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:52 2022] PUT /identity/v3/projects/42f936d2a52945ffb0fb540539d36e36/users/641543d9429a405c8e435ca344ea5fe3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:52.489032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0a13-ff22-4e05-8a04-56bff6564582 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.489523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0a13-ff22-4e05-8a04-56bff6564582 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.490014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f0a13-ff22-4e05-8a04-56bff6564582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.539752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:52.547136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-c25f0a13-ff22-4e05-8a04-56bff6564582 None None] MFA Rules not processed for user `641543d9429a405c8e435ca344ea5fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:52.551611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c25f0a13-ff22-4e05-8a04-56bff6564582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:52.724084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1176/2353] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:52.740147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:52.741867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:52.893573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.894849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.895957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.918919 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cc5f-ed6a-4701-8726-62824f96b60a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:52.924802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:52.925655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:52.926101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:52.926529 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:52.940449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:52.941037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:52.942695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:52.943039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:52.961088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1178/2354] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:53.061681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1177/2355] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:53.090045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:53.091732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:53.179981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:53.205805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664e1e-2f1b-43c1-8d79-bad13a1e5e26 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:53.214415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:53.215943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:53.216328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:53.216716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:53.227447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:53.229771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f151-8d23-4f49-966b-1bf8054069f4 service swift] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:53.242271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1179/2356] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:58:53.715187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:53.950779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638a31b-9c61-425c-806a-9c91b325c4ef None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:53.967949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:53.970560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:53.970676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:53.973975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] PATH_INFO: `/v3/users/8ef54e01a8a6413e809abe8f9427da14` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:53.976410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef54e01a8a6413e809abe8f9427da14)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:53.978500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:54.082959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:54.086043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef54e01a8a6413e809abe8f9427da14'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:54.088530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef54e01a8a6413e809abe8f9427da14'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:54.089763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef54e01a8a6413e809abe8f9427da14'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:54.105592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef54e01a8a6413e809abe8f9427da14'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:54.127786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef54e01a8a6413e809abe8f9427da14'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:54.146495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1178/2357] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:58:53 2022] DELETE /identity/v3/users/8ef54e01a8a6413e809abe8f9427da14 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:54.151394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:54.152252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:54.152252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:54.180588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:54.181986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:54.313408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1180/2358] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:58:54.327192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:54.344741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:54.461947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b1319-b7ff-4390-89a6-2a36249f7be8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:54.467873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:54.468588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:54.468860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:54.469158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:54.480541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:54.482131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:54.521304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1179/2359] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:58:54.923108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:54.925659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:54.948481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b8e6e-d612-4566-b452-7d4b375d9799 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:54.955006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:54.956302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:54.957041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:54.957441 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] PATH_INFO: `/v3/projects/3d09f7144d554e2d8a1c2e0177f3a81b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:54.960334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d09f7144d554e2d8a1c2e0177f3a81b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:54.964026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:55.087037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d09f7144d554e2d8a1c2e0177f3a81b', 'request_id': 'req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.102773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d09f7144d554e2d8a1c2e0177f3a81b', 'request_id': 'req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.132378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d09f7144d554e2d8a1c2e0177f3a81b', 'request_id': 'req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.156470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] The token cache is being invalidate because project 3d09f7144d554e2d8a1c2e0177f3a81b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:58:55.157600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.158581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.162105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1181/2360] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:58:54 2022] DELETE /identity/v3/projects/3d09f7144d554e2d8a1c2e0177f3a81b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:55.166989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:55.226771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96bad6-93b0-47ff-97fc-29653a59a774 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:55.380556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:55.381734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:55.382204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:55.382674 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] PATH_INFO: `/v3/users/641543d9429a405c8e435ca344ea5fe3` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:55.385189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641543d9429a405c8e435ca344ea5fe3)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:55.389211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:55.535924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641543d9429a405c8e435ca344ea5fe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.537585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641543d9429a405c8e435ca344ea5fe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.538758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641543d9429a405c8e435ca344ea5fe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.557358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641543d9429a405c8e435ca344ea5fe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.572692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:55.580448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-3be28235-196a-40da-b229-9d0439599126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641543d9429a405c8e435ca344ea5fe3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:55.605941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1180/2361] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:58:55 2022] DELETE /identity/v3/users/641543d9429a405c8e435ca344ea5fe3 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:56.052836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:56.054543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:56.074601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd51f0-31e9-4246-a0b7-5ada1ffd7605 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:56.271778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:56.273172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:56.273666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:56.274147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] PATH_INFO: `/v3/projects/42f936d2a52945ffb0fb540539d36e36` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:56.277875 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f936d2a52945ffb0fb540539d36e36)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:56.282907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:56.303151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:56.470440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f936d2a52945ffb0fb540539d36e36', 'request_id': 'req-d1eb55bf-d880-475f-9d77-689a4fa74584'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:56.483030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f936d2a52945ffb0fb540539d36e36', 'request_id': 'req-d1eb55bf-d880-475f-9d77-689a4fa74584'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:56.494298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f936d2a52945ffb0fb540539d36e36', 'request_id': 'req-d1eb55bf-d880-475f-9d77-689a4fa74584'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:56.521688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] The token cache is being invalidate because project 42f936d2a52945ffb0fb540539d36e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:58:56.522770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:56.524272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d1eb55bf-d880-475f-9d77-689a4fa74584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:58:56.528007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1182/2362] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:58:56 2022] DELETE /identity/v3/projects/42f936d2a52945ffb0fb540539d36e36 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:56.535881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:56.539109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:56.540779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:56.589392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:56.592429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:56.719348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1181/2363] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:58:56.721376 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:56.721738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:56.722087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:56.750043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:56.752472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:56.781912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1183/2364] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:56.783363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:56.797946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c020ba-5c42-4c03-8f87-d4efa72d8c5b None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:56.806737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:56.807599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:56.807873 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:56.808264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:56.809073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:58:56.809452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:56.811533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:56.832736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1182/2365] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:58:56.834072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:56.848207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedae68-8f47-4daa-b867-419a93be03e0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:56.853677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:56.854319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:56.854600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:56.854872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:56.855574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:56.857547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:56.898943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1184/2366] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:58:56 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:56.904964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:56.939713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a2b45-eede-453c-b0c8-a5c48a4e332c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.095829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:57.099451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:57.169933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.172346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.172346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.172800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.174284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.182365 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:57.242263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1183/2367] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:58:56 2022] POST /identity/v3/users => generated 282 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:57.248418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:57.285503 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007a95a-d893-4852-974a-41bfae242454 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.292220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.296252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.299965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.302056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.306072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.311346 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:57.314251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d67b4-9acc-45bb-99b9-062a6c974ee1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.325209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.327672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.328263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.328907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.334364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:57.346421 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1184/2368] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:57.348635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:57.362844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.366824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:57.371899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d2360-172f-4ceb-9ee7-734f89e30f93 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.402949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.404686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.405169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.405539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.406413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.408488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc24b6a-650b-4024-863f-719242323381 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:57.424226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1185/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:58:57.430224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1185/2370] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:57.430443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:57.480704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef90a8-8f54-4606-9901-163db8d985f8 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.494641 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.497850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.498238 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.498616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] PATH_INFO: `/v3/projects/2fda3ca9e64346b29f3322539ec61a56/users/ade0a1aabaef4e6590dba808c5306a03/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.541782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fda3ca9e64346b29f3322539ec61a56, user_id=ade0a1aabaef4e6590dba808c5306a03, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.545631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:57.571183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1186/2371] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:57 2022] PUT /identity/v3/projects/2fda3ca9e64346b29f3322539ec61a56/users/ade0a1aabaef4e6590dba808c5306a03/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:57.585126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.585742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.586088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.634707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] MFA Rules not processed for user `ade0a1aabaef4e6590dba808c5306a03`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:57.647584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:57.714270 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:57.820802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1186/2372] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:58:57.825704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:57.866818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:57.928072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb88ef-557b-4784-a058-d2acfbc7ee79 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:57.938234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:57.939181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:57.939656 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:57.940233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:57.941155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:57.943521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.014800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1187/2373] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:58:57 2022] POST /identity/v3/projects => generated 363 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:58.017565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.034497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8f62-9c48-4b28-9331-7a12949ae4a3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.198955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.203096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.204148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.204826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.206187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.211452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.293333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.302101 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1187/2374] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:58:58 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:58:58.312514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7a85-56bc-4c87-a2d4-c3df18254bbc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.318242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.318910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.319341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.319679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.320405 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.322692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.339886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.346538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1188/2375] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:58.352235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513c63b-f536-4fd3-b093-e53d49eba257 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.358753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.360661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.361265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.361780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] PATH_INFO: `/v3/projects/0817d5cc93604b07bd03636454dce88d/users/ee93dbbf669f43c2b792d2f61010bd6b/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.387836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0817d5cc93604b07bd03636454dce88d, user_id=ee93dbbf669f43c2b792d2f61010bd6b, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.390730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.427721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1188/2376] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:58 2022] PUT /identity/v3/projects/0817d5cc93604b07bd03636454dce88d/users/ee93dbbf669f43c2b792d2f61010bd6b/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:58.434267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.452162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2287f-7ede-4b6f-be96-b506bd3ec541 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.610334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.611387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.611768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.612167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.612949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.615146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6f176-69f4-4f91-a952-2a2e4b9d7609 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.636213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1189/2377] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:58:58.637911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.651492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca857-02df-4a45-a776-0e1152843e3b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.657952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.658793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.659347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.659730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] PATH_INFO: `/v3/projects/0817d5cc93604b07bd03636454dce88d/users/ee93dbbf669f43c2b792d2f61010bd6b/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.662011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0817d5cc93604b07bd03636454dce88d, user_id=ee93dbbf669f43c2b792d2f61010bd6b, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.665376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:58.684328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1189/2378] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:58:58 2022] PUT /identity/v3/projects/0817d5cc93604b07bd03636454dce88d/users/ee93dbbf669f43c2b792d2f61010bd6b/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:58:58.696506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.697194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.697577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.745769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] MFA Rules not processed for user `ee93dbbf669f43c2b792d2f61010bd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:58:58.747759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:58.852140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1190/2379] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:58:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:58:58.865564 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:58.867534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:58:58.972590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c922345-ac63-49bb-827f-8f12238cf305 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:58.976888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:58.978036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:58.978480 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:58.978902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:58.992077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:58.993864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593aa6d-713f-43c5-bb8d-87fb949cf5ad service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:59.008859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1190/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:58:59.157660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:58:59.175523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4bddf-e51f-4139-9980-b60e1a1fe25f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:58:59.185757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:58:59.187016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:58:59.187669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:58:59.188335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:58:59.225629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:58:59.304377 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:58:59.306978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:58:59.330868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1191/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:58:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:01.839147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487593-f141-4fd1-a126-3480a2bdde72 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:01.840291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487593-f141-4fd1-a126-3480a2bdde72 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:01.840634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487593-f141-4fd1-a126-3480a2bdde72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:01.865030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-b0487593-f141-4fd1-a126-3480a2bdde72 None None] MFA Rules not processed for user `0d81b048f7e64c53894ad9e9ee33f518`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:01.866482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b0487593-f141-4fd1-a126-3480a2bdde72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:01.892354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:01.961224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1191/2382] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:09.323543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=18 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:09.687414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:09.779490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69829803-2ef5-4eb9-82ce-af1a0563af95 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:09.785669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:09.786343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:09.786622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:09.786893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] PATH_INFO: `/v3/users/ade0a1aabaef4e6590dba808c5306a03` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:09.788282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade0a1aabaef4e6590dba808c5306a03)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:09.790706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:09.865053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade0a1aabaef4e6590dba808c5306a03'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:09.865899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade0a1aabaef4e6590dba808c5306a03'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:09.875411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade0a1aabaef4e6590dba808c5306a03'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:09.889703 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade0a1aabaef4e6590dba808c5306a03'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:09.932802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade0a1aabaef4e6590dba808c5306a03'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:09.969008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1192/2383] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:09 2022] DELETE /identity/v3/users/ade0a1aabaef4e6590dba808c5306a03 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:09.970755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:09.971276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:09.971366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:10.006199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:10.010590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:10.129510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1192/2384] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:10.145498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:10.148165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:10.256225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fb552-0d7f-4d99-b98b-9b875bf7929d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:10.262223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:10.263305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:10.263798 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:10.264211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:10.281728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:10.284762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:10.302435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1193/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:10.415872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:10.417320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:10.527190 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213c590-4b5d-4db4-8883-95f6d69bbb42 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:10.537051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:10.537746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:10.538246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:10.538565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:10.554195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:10.580017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:10.582028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:10.597116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1193/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:10.739130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:10.740792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:10.760577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5187-dc89-4b36-a9e6-a39fda28ddf0 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:10.771370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:10.771855 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:10.772118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:10.772398 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] PATH_INFO: `/v3/projects/2fda3ca9e64346b29f3322539ec61a56` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:10.773849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fda3ca9e64346b29f3322539ec61a56)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:10.776095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:10.799746 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:10.850319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fda3ca9e64346b29f3322539ec61a56', 'request_id': 'req-b04747d1-0fc0-423d-a742-deebfdf2bb1f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:10.862311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fda3ca9e64346b29f3322539ec61a56', 'request_id': 'req-b04747d1-0fc0-423d-a742-deebfdf2bb1f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:10.871645 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fda3ca9e64346b29f3322539ec61a56', 'request_id': 'req-b04747d1-0fc0-423d-a742-deebfdf2bb1f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:10.901734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] The token cache is being invalidate because project 2fda3ca9e64346b29f3322539ec61a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:10.902017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:10.902795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:10.904676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1194/2387] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:10 2022] DELETE /identity/v3/projects/2fda3ca9e64346b29f3322539ec61a56 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:10.909042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:10.921321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b6dc4-318d-473a-aa8a-13308e2cdb4b service nova] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:10.980968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:10.989924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:10.990745 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:10.991045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:10.991463 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] PATH_INFO: `/v3/users/ee93dbbf669f43c2b792d2f61010bd6b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:10.993107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee93dbbf669f43c2b792d2f61010bd6b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:10.995076 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:11.077186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee93dbbf669f43c2b792d2f61010bd6b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.077991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee93dbbf669f43c2b792d2f61010bd6b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.078811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee93dbbf669f43c2b792d2f61010bd6b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.087509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee93dbbf669f43c2b792d2f61010bd6b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.102614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-53cafcae-bf8c-4c2a-b066-1f02a0f2e74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee93dbbf669f43c2b792d2f61010bd6b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.118470 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1194/2388] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:10 2022] DELETE /identity/v3/users/ee93dbbf669f43c2b792d2f61010bd6b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:11.584107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:11.585813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:11.600995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04747d1-0fc0-423d-a742-deebfdf2bb1f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:11.668964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:11.669630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:11.669901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:11.670174 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] PATH_INFO: `/v3/projects/0817d5cc93604b07bd03636454dce88d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:11.671436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0817d5cc93604b07bd03636454dce88d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:11.673379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:11.741187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0817d5cc93604b07bd03636454dce88d', 'request_id': 'req-3a29c498-6600-4ce4-baa3-62b0cb5003c0'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.748935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0817d5cc93604b07bd03636454dce88d', 'request_id': 'req-3a29c498-6600-4ce4-baa3-62b0cb5003c0'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.757399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0817d5cc93604b07bd03636454dce88d', 'request_id': 'req-3a29c498-6600-4ce4-baa3-62b0cb5003c0'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.773982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] The token cache is being invalidate because project 0817d5cc93604b07bd03636454dce88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:11.774338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.775124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3a29c498-6600-4ce4-baa3-62b0cb5003c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:11.777630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1195/2389] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:11 2022] DELETE /identity/v3/projects/0817d5cc93604b07bd03636454dce88d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:11.783838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:11.784150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:11.784423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:11.807935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:11.809063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:11.873224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1195/2390] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:11.881601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:11.881952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:11.882332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:11.905220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:11.906568 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:11.915500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1196/2391] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:11.920110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:11.930799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08f4e7-2f7a-44a8-a25c-21e08b765ac5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:11.936431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:11.937071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:11.937343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:11.937661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:11.938245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:59:11.938571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:11.940217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:11.952918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1196/2392] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:59:11.957361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:11.968925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007eb56-4c24-47e5-8fa0-9e91119d27ec None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:11.973812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:11.974431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:11.974714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:11.974985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:11.975666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:11.977780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.009620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1197/2393] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:11 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:12.014251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.026481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd9c21-191c-4bfe-b47a-77d46f1dc8db admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.094082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.094842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.095323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.095626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.096458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.098603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.114990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:12.124907 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1197/2394] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:12.129986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.142727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95b550-76a6-4b7f-b478-cce8a0269fb9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.148075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.148689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.148968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.149249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.149889 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.151518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.153182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:12.164918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1198/2395] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:12.168743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.179013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3120-32de-49c8-bd57-6f8a42250c48 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.193551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.193551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.193551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.193551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.193929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.193929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e763943-7fc8-4e6d-a96e-5137e8700138 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.202293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1198/2396] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:59:12.206799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.216832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18639f2a-aee5-4bc5-9e20-9e5507c0bade admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.221570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.222221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.222518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.222771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] PATH_INFO: `/v3/projects/e74a245fb6684dde9caff2376e288499/users/f794fa9a2a9647a8a1add70f784503eb/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.239045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74a245fb6684dde9caff2376e288499, user_id=f794fa9a2a9647a8a1add70f784503eb, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.241942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.258263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1199/2397] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:12 2022] PUT /identity/v3/projects/e74a245fb6684dde9caff2376e288499/users/f794fa9a2a9647a8a1add70f784503eb/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:12.270538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.270822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.271157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.308267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] MFA Rules not processed for user `f794fa9a2a9647a8a1add70f784503eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:12.310395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:12.413244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1199/2398] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:12.419804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.505491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbd831-dc5a-4c82-bcca-ae80bd9165bb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.511162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.511918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.512283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.512736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.513489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.515702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.553731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1200/2399] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:12.559665 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.578899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85959-9436-4df0-b6d9-d4c840798969 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.628106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.630392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:12.659975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.660793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.661317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.661756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.663276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.665795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.701508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1200/2400] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:12.703235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.715310 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858767e-f6a5-428d-9283-1d840e320e57 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.720852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.721490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.721767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.722034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.723798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248595f-0faf-4c12-bdf6-da43327613f8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.724103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:12.730093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.731099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.731513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.731946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.732924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.734732 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.742194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.745584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.751044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1201/2401] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:12.755638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.759806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1201/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:12.766794 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d9980-024d-4296-ba53-1841e4e8a32c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.772668 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.773712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.773859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.774049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] PATH_INFO: `/v3/projects/ae2433f8242f4c1ca68432de48c65957/users/388cc2cf0b6f4d989a2a42ae023eb85a/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.793373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2433f8242f4c1ca68432de48c65957, user_id=388cc2cf0b6f4d989a2a42ae023eb85a, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.796456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc3bfa-d592-469b-a874-0e6394e35f06 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.814440 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1202/2403] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:12 2022] PUT /identity/v3/projects/ae2433f8242f4c1ca68432de48c65957/users/388cc2cf0b6f4d989a2a42ae023eb85a/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:12.818843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.832175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce714ef-c93f-4ae8-be25-1bcdeefd8c9e service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.918834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e77bd-8d7c-4861-94f2-a698849b6728 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.919521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e77bd-8d7c-4861-94f2-a698849b6728 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.920017 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e77bd-8d7c-4861-94f2-a698849b6728 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.920351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.921124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.921410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.921675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.922336 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.923112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1203/2404] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 12 23:59:12.924318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.926040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:12.935350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.936113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.936846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.939702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1202/2405] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:12.942382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:12.953612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40d100-6ab4-4b5c-a586-fc533a449b33 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:12.960475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:12.961640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.961922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.962374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] PATH_INFO: `/v3/projects/ae2433f8242f4c1ca68432de48c65957/users/388cc2cf0b6f4d989a2a42ae023eb85a/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:12.962699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:12.964409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:12.964578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2433f8242f4c1ca68432de48c65957, user_id=388cc2cf0b6f4d989a2a42ae023eb85a, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:12.967700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc8029-ae6b-4ada-8a67-8af0090e17e8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:12.982576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1203/2406] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:12 2022] PUT /identity/v3/projects/ae2433f8242f4c1ca68432de48c65957/users/388cc2cf0b6f4d989a2a42ae023eb85a/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:12.992337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:12.994299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:12.994747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:12.995348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.031493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] MFA Rules not processed for user `388cc2cf0b6f4d989a2a42ae023eb85a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:13.032997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:13.058780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1204/2407] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:13.077721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:13.079559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:13.125564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1204/2408] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:13.136804 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:13.138523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:13.167810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4fa2b-7b2c-4e53-9b72-1bcb3cf88137 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:13.220956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daea109-b16c-4a3d-910e-c023b2a8c98c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:13.226546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:13.227440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.227813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.228222 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.240327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:13.241930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebd8f6-2a8d-422b-bfbd-db4f45a41de6 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:13.245194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:13.245820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.246092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.246360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.254064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1205/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:13.385057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:13.386915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bde845-c287-4ea8-a712-284ce63764b7 service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:13.403896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1205/2410] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:13.489768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82c07-f942-4750-8d1b-ffcc726f3392 None None] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.490052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82c07-f942-4750-8d1b-ffcc726f3392 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.490352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be82c07-f942-4750-8d1b-ffcc726f3392 None None] PATH_INFO: `/` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.491773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1206/2411] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:59:13.500894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb58de-0200-4363-9dba-09364aced5cd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.501288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb58de-0200-4363-9dba-09364aced5cd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.501724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb58de-0200-4363-9dba-09364aced5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.527424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7eeb58de-0200-4363-9dba-09364aced5cd None None] MFA Rules not processed for user `e9c49b9be98a4a9194239c5be186e1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:13.528947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7eeb58de-0200-4363-9dba-09364aced5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:13.538915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:13.592819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1206/2412] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:59:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:13.608958 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be82c07-f942-4750-8d1b-ffcc726f3392 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:13.625760 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be82c07-f942-4750-8d1b-ffcc726f3392 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:13.631483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:13.632592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.632878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.633232 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.643263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:13.644805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cecf1a-1147-4cac-99ca-41b57c2605ad service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:13.654426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1207/2413] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:13.747445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d36de0-60ae-46b8-a40f-ed6674ca78a5 None None] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.747740 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d36de0-60ae-46b8-a40f-ed6674ca78a5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.748059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d36de0-60ae-46b8-a40f-ed6674ca78a5 None None] PATH_INFO: `/` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.750289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1207/2414] 10.222.0.26 () {54 vars in 921 bytes} [Mon Sep 12 23:59:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 12 23:59:13.757317 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.757937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.758173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.764523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d36de0-60ae-46b8-a40f-ed6674ca78a5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:13.780250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] MFA Rules not processed for user `97e8cd3ae37443acbb2c7c710646686a`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:13.781674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:13.791743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1208/2415] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Sep 12 23:59:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:13.805406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:13.819079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d825f-6670-4bce-ba13-71051fdaa56a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:13.823863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:13.824487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.824807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.825282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.838646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:13.840010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d36de0-60ae-46b8-a40f-ed6674ca78a5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:13.840288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e20c2-9060-4776-bf78-60f509f4129f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:13.845219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:13.845965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.846349 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.846680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] PATH_INFO: `/v3/users/f794fa9a2a9647a8a1add70f784503eb` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.848721 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f794fa9a2a9647a8a1add70f784503eb)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:13.850754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:13.852088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1209/2416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:13.913290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f794fa9a2a9647a8a1add70f784503eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:13.914026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f794fa9a2a9647a8a1add70f784503eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:13.915453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f794fa9a2a9647a8a1add70f784503eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:13.922902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f794fa9a2a9647a8a1add70f784503eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:13.935838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f794fa9a2a9647a8a1add70f784503eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:13.950319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1208/2417] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:13 2022] DELETE /identity/v3/users/f794fa9a2a9647a8a1add70f784503eb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:13.955131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:13.955510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:13.955849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:13.976880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:13.978181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:14.024442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:14.067745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1210/2418] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:14.080330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:14.081909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:14.142614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:14.161305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1da8c-e4b9-44c6-ae88-d5245663995b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:14.165641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:14.166353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:14.166543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:14.166809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:14.177220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:14.178666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:14.188905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1209/2419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:14.584613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:14.586461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:14.600079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949286a8-b2c4-4615-ba3a-3990bff89886 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:14.609521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:14.610146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:14.610439 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:14.610713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] PATH_INFO: `/v3/projects/e74a245fb6684dde9caff2376e288499` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:14.630606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74a245fb6684dde9caff2376e288499)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:14.633730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:14.703927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74a245fb6684dde9caff2376e288499', 'request_id': 'req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.716961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74a245fb6684dde9caff2376e288499', 'request_id': 'req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.723997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74a245fb6684dde9caff2376e288499', 'request_id': 'req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.741016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] The token cache is being invalidate because project e74a245fb6684dde9caff2376e288499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:14.741331 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.741937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.743524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1211/2420] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:14 2022] DELETE /identity/v3/projects/e74a245fb6684dde9caff2376e288499 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:14.747727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:14.757911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d44f7-ca50-4016-b312-4d6c293acf7c service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:14.826834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:14.827479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:14.827759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:14.828043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] PATH_INFO: `/v3/users/388cc2cf0b6f4d989a2a42ae023eb85a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:14.829238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=388cc2cf0b6f4d989a2a42ae023eb85a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:14.831489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:14.890984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388cc2cf0b6f4d989a2a42ae023eb85a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.891727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388cc2cf0b6f4d989a2a42ae023eb85a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.892376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388cc2cf0b6f4d989a2a42ae023eb85a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.901152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388cc2cf0b6f4d989a2a42ae023eb85a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.914771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-adb38c72-2ed7-45af-a386-83e9b80286bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388cc2cf0b6f4d989a2a42ae023eb85a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:14.932096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1210/2421] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:14 2022] DELETE /identity/v3/users/388cc2cf0b6f4d989a2a42ae023eb85a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:15.237109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.238304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:15.259243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30c8b9-a8e0-478d-b1ef-0355bc14556d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.323190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.323865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.324139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.324408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] PATH_INFO: `/v3/projects/ae2433f8242f4c1ca68432de48c65957` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.325520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2433f8242f4c1ca68432de48c65957)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.327517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.381909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2433f8242f4c1ca68432de48c65957', 'request_id': 'req-80b9529b-d957-463e-9a36-6b86555741af'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:15.389248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2433f8242f4c1ca68432de48c65957', 'request_id': 'req-80b9529b-d957-463e-9a36-6b86555741af'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:15.397527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2433f8242f4c1ca68432de48c65957', 'request_id': 'req-80b9529b-d957-463e-9a36-6b86555741af'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:15.399500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:15.414688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] The token cache is being invalidate because project ae2433f8242f4c1ca68432de48c65957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:15.414953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:15.415730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-80b9529b-d957-463e-9a36-6b86555741af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:15.417253 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1212/2422] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:15 2022] DELETE /identity/v3/projects/ae2433f8242f4c1ca68432de48c65957 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:15.422163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.422648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.422742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.443500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:15.444737 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:15.507776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1211/2423] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:15.516807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.517060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.517359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.538965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:15.540218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:15.549112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1213/2424] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:15.553635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.559980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:15.564373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b841f-49a3-4aa1-b3d2-790ae8e8d8bc None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.570573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.571176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.571545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.571822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.572697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:59:15.573202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.575417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.587849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1212/2425] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:59:15.592435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.602280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b812c28-bbaf-46da-9bb1-45c4c966850d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.607498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.608138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.608420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.608686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.609313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.611652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.647300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1214/2426] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:15.650296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.660716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53370ff-d2ec-4fff-ae83-b5cd688cb185 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.720904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.721708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.721982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.722246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.722889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.728227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff90bc-ab16-4c4a-be81-fcdbd4991679 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.759774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1213/2427] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:15.765815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.788519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ae5d-33e3-4220-a20c-a20da2df300b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.798289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.798536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.798536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.799033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.799966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.800210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.800608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.801545 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.803507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.828905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1215/2428] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:15.830241 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.831883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:15.833191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:15.851372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da9378-bb35-49d4-ae3b-306777900c1a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.863891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.864987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.865473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.865997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.867061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.870258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.897578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1216/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:15.899429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.908562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1214/2430] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:15.921575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68711c-46d6-4d03-92c4-6a098e69a2d4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:15.928841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:15.929621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.930000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.930390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] PATH_INFO: `/v3/projects/16457deab6cb4c3e871ce7ac2c8fa799/users/1dec7a8ff84148aca87f818c58cda05b/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:15.946670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:15.948267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:15.953638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16457deab6cb4c3e871ce7ac2c8fa799, user_id=1dec7a8ff84148aca87f818c58cda05b, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:15.957140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3ecb-443e-4dc5-afcc-5d34bd189bf5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:15.991272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1217/2431] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:15 2022] PUT /identity/v3/projects/16457deab6cb4c3e871ce7ac2c8fa799/users/1dec7a8ff84148aca87f818c58cda05b/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:15.997223 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:15.998025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:15.998706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.053978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] MFA Rules not processed for user `1dec7a8ff84148aca87f818c58cda05b`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:16.055363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:16.058312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92d3a3-ab75-49ca-b05f-41ad6b098673 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:16.134246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:16.160583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1218/2432] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:16.181060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:16.182975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:16.186749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:16.187912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:16.187912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:16.188230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.253335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316773c-0257-40a0-9938-7f604e8948d8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:16.257474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:16.258115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:16.258433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:16.258708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.265618 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:16.267454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:16.270190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:16.271922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc957e-14d5-4291-a4a5-c0ada2ab79ef service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:16.277358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1215/2433] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:59:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 12 23:59:16.285948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1219/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:16.310764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:16.412015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24017-461f-45c5-9a41-bf42b458b935 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:16.429196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:16.429908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:16.430158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:16.430434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] PATH_INFO: `/v3/users/1dec7a8ff84148aca87f818c58cda05b` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.431992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dec7a8ff84148aca87f818c58cda05b)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:16.433989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:16.531348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dec7a8ff84148aca87f818c58cda05b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:16.532388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dec7a8ff84148aca87f818c58cda05b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:16.533255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dec7a8ff84148aca87f818c58cda05b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:16.543081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dec7a8ff84148aca87f818c58cda05b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:16.571614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:16.573855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dec7a8ff84148aca87f818c58cda05b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:16.609577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1216/2435] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:16 2022] DELETE /identity/v3/users/1dec7a8ff84148aca87f818c58cda05b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:16.619805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:16.620518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:16.621066 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.658572 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:16.660239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:16.665837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:16.763056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1220/2436] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:16.775904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:16.777595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:16.857068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b2335-5e3f-4fad-a9e4-f90d3d8f65e9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:16.860925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:16.861707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:16.862059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:16.862391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:16.872629 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:16.874192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cecfb0-6c2e-40a8-a1ab-573189881f82 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:16.886728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1217/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:17.442622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:17.444649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:17.476638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ab106-ac89-479b-8c66-6940d7fcfbcd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:17.483792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:17.484450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:17.484726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:17.484996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] PATH_INFO: `/v3/projects/16457deab6cb4c3e871ce7ac2c8fa799` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:17.486225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16457deab6cb4c3e871ce7ac2c8fa799)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:17.488940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:17.554794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16457deab6cb4c3e871ce7ac2c8fa799', 'request_id': 'req-7467476f-1edd-487f-b758-9f4efa6ee292'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:17.562207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16457deab6cb4c3e871ce7ac2c8fa799', 'request_id': 'req-7467476f-1edd-487f-b758-9f4efa6ee292'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:17.569863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16457deab6cb4c3e871ce7ac2c8fa799', 'request_id': 'req-7467476f-1edd-487f-b758-9f4efa6ee292'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:17.585587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] The token cache is being invalidate because project 16457deab6cb4c3e871ce7ac2c8fa799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:17.586141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:17.587077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7467476f-1edd-487f-b758-9f4efa6ee292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:17.589401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1221/2438] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:17 2022] DELETE /identity/v3/projects/16457deab6cb4c3e871ce7ac2c8fa799 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:17.596271 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:17.597299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:17.597887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:17.625531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:17.627342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:17.738633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1218/2439] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:17.746506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:17.746796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:17.747049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:17.777051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:17.778312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:17.788031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1222/2440] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:17.793468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:17.804624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af340516-a227-4406-a2a0-da757acd4dfb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:17.811047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:17.811727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:17.811999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:17.812424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:17.813128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:59:17.813487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:17.815312 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:17.835783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1219/2441] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:59:17.836842 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:17.849104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d722-d9f6-4acc-866e-93c502ccb102 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:17.854866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:17.855532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:17.855805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:17.856059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:17.856707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:17.858657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:17.896715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1223/2442] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:17 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:17.898502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:17.916257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8a76a-24a2-4bbf-b408-4e79197e54d9 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:17.999626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.000235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.000589 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.000977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.001708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.003802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.037982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1220/2443] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:17 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:18.042493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.061751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f38c7d-6556-4923-be89-e38ce67b99ef admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.076512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.077639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.078135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.078559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.079401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.081490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.103117 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1224/2444] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:59:18.104456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.115250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f415c6-289e-4b61-ae04-265c2942a054 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.120734 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.121368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.121666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.121941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.122621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.124284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c301e-ede4-4a4f-84ac-c15180860975 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.127760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:18.143288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.144219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1221/2445] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:18.159564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489f67e-4381-4ccc-8566-79c4ab11d3cc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.165953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.166659 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.166944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.167245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] PATH_INFO: `/v3/projects/fc0569bab5414c4b88d2b9a82a13ceec/users/d7f86147974f400a8b0a7d7f50da9a15/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.185674 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0569bab5414c4b88d2b9a82a13ceec, user_id=d7f86147974f400a8b0a7d7f50da9a15, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.188460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.205706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1225/2446] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:18 2022] PUT /identity/v3/projects/fc0569bab5414c4b88d2b9a82a13ceec/users/d7f86147974f400a8b0a7d7f50da9a15/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:18.218097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.218424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.219149 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.255806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] MFA Rules not processed for user `d7f86147974f400a8b0a7d7f50da9a15`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:18.257012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:18.353023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1222/2447] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:18.357993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.427510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:18.531859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe51b7a-2a39-4b53-9943-7e535410ff32 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.552974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.555817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.555817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.555817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.557101 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.563827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.620381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1226/2448] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:18 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:18.634423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.652761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7bcce-c26e-4d2c-9ab6-8b6e3cda6c59 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.813120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.813895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.814176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.814569 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.815727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.820478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.853141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1223/2449] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:18 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:18.857356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.882352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4970d-a213-4d24-9a49-8f6faf596aa8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.899643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.900634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.900985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.901452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.902500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.904766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.917633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1227/2450] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:18.921867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.934502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712d9a0-cd46-4038-8a2c-536e6f46c58b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.941107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.942145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.942695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.943185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:18.944680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:18.946727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b04bba-844f-47b9-9169-e196c884e276 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:18.959924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1224/2451] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:18.964320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:18.975782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005950cd-d74c-4007-b785-e8b0543123df admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:18.981432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:18.982899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:18.983201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:18.983916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] PATH_INFO: `/v3/projects/42e6518b137f4fb086720ac8747428d2/users/f9f9c94012694a3f8264394f6fd0116c/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.009416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e6518b137f4fb086720ac8747428d2, user_id=f9f9c94012694a3f8264394f6fd0116c, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.012356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.031616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1228/2452] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:18 2022] PUT /identity/v3/projects/42e6518b137f4fb086720ac8747428d2/users/f9f9c94012694a3f8264394f6fd0116c/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:19.045431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.046328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.046910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.084432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] MFA Rules not processed for user `f9f9c94012694a3f8264394f6fd0116c`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:19.085829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:19.169976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:19.190180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1225/2453] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:19.198866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.306733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a1069-7caa-4571-9699-de226701d2c9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.312859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.313616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.313899 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.314173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.314888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.317123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.353467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1229/2454] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:19 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:19.358032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.370187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe78f61-b91f-455e-a87f-70a6cadbbd15 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.445305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.445972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.446243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.446513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.447160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.449191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.478926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.484789 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1226/2455] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:19 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:19.490977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986fc21-98bc-4619-85ba-94f43ee2fddc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.496135 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.496768 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.497081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.497353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.498028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.499795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.512321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1230/2456] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:19.520779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.535834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d25ed-2ce0-4cf4-9366-08af1983c23a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.542103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.542937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.543212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.543516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] PATH_INFO: `/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde/users/ded937513d8d4e9a8321df84821b131b/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.562671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a18ad0fcb14c938840b2a93f94ffde, user_id=ded937513d8d4e9a8321df84821b131b, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.565775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.585378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1227/2457] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:19 2022] PUT /identity/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde/users/ded937513d8d4e9a8321df84821b131b/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:19.586365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.597664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18a2c7-e1a2-4c6c-a14a-856a2896111a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.667502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:19.680850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.681728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.682040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.682389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.683183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.685408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bb062-df8d-4a22-b427-14b1b8a66dca admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.701396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1231/2458] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:19.702793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:19.715950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa4f11-51fe-47d5-843f-e531f880ed93 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:19.725889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:19.727363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.728221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.728221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] PATH_INFO: `/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde/users/ded937513d8d4e9a8321df84821b131b/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.732431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a18ad0fcb14c938840b2a93f94ffde, user_id=ded937513d8d4e9a8321df84821b131b, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:19.737408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:19.774088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1228/2459] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:19 2022] PUT /identity/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde/users/ded937513d8d4e9a8321df84821b131b/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:19.800996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:19.801449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:19.801845 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:19.863684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] MFA Rules not processed for user `ded937513d8d4e9a8321df84821b131b`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:19.865021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:20.001649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1232/2460] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:20.014381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:20.016386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:20.266558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca098c5-0469-45ec-8585-c6816245c4a1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:20.270605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:20.271246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:20.271531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:20.271805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:20.340339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:20.342301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:20.359322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1229/2461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 12 23:59:22.065782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:22.067491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:22.082460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb1373-7d37-4ef0-9612-6cc623730922 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:22.095083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:22.096258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:22.097363 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:22.097791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:22.184070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:22.186913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:22.207332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1233/2462] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:22.957727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:22.962381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:22.968760 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580fd3-553c-49a7-9927-3daf83e6bba6 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:22.973763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:22.974314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:22.974583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:22.974902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:22.987735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:22.989321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe8cbc-617b-45c2-80d3-b66056cf62ba service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:23.001323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1230/2463] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:27.101687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:27.166358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cda09-a816-4e58-8232-9d527d76fdf5 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:27.171699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:27.172339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:27.172689 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:27.172956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] PATH_INFO: `/v3/users/d7f86147974f400a8b0a7d7f50da9a15` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:27.174113 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f86147974f400a8b0a7d7f50da9a15)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:27.176109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:27.190809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:27.228484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f86147974f400a8b0a7d7f50da9a15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.229103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f86147974f400a8b0a7d7f50da9a15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.236608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f86147974f400a8b0a7d7f50da9a15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.244977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f86147974f400a8b0a7d7f50da9a15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.258953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f86147974f400a8b0a7d7f50da9a15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.265294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1234/2464] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:27 2022] DELETE /identity/v3/users/d7f86147974f400a8b0a7d7f50da9a15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:27.270845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:27.272654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:27.272654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:27.291723 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:27.292968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:27.381342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1231/2465] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:27.393757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:27.395579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:27.489682 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb34f-5a38-46bf-880b-a845faeb0510 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:27.493418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:27.494057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:27.494334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:27.494602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:27.504846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:27.506312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:27.517807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1235/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:27.837901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:27.839516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:27.857472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac51ae1-4a6c-454d-b680-6ca488e0c0bf None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:27.864458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:27.865318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:27.865691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:27.866081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] PATH_INFO: `/v3/projects/fc0569bab5414c4b88d2b9a82a13ceec` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:27.867430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0569bab5414c4b88d2b9a82a13ceec)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:27.870361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:27.960783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0569bab5414c4b88d2b9a82a13ceec', 'request_id': 'req-658f799a-a386-4b92-9638-80a21ed7e18e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.968437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0569bab5414c4b88d2b9a82a13ceec', 'request_id': 'req-658f799a-a386-4b92-9638-80a21ed7e18e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.979746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0569bab5414c4b88d2b9a82a13ceec', 'request_id': 'req-658f799a-a386-4b92-9638-80a21ed7e18e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.996135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] The token cache is being invalidate because project fc0569bab5414c4b88d2b9a82a13ceec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:27.996358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.997097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:27.998774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1232/2467] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:27 2022] DELETE /identity/v3/projects/fc0569bab5414c4b88d2b9a82a13ceec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:28.003108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:28.015262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e38bae-2237-4ba2-ae9f-d9e4eab4c41e service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:28.107841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:28.108532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:28.108748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:28.109264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] PATH_INFO: `/v3/users/f9f9c94012694a3f8264394f6fd0116c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:28.110618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9f9c94012694a3f8264394f6fd0116c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:28.112794 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:28.178787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:28.192558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9f9c94012694a3f8264394f6fd0116c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.193967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9f9c94012694a3f8264394f6fd0116c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.224460 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9f9c94012694a3f8264394f6fd0116c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.240748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9f9c94012694a3f8264394f6fd0116c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.270389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9f9c94012694a3f8264394f6fd0116c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.278079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1236/2468] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:28 2022] DELETE /identity/v3/users/f9f9c94012694a3f8264394f6fd0116c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:28.648982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:28.650770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:28.666170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f799a-a386-4b92-9638-80a21ed7e18e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:28.735300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:28.735985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:28.736260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:28.736529 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] PATH_INFO: `/v3/projects/42e6518b137f4fb086720ac8747428d2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:28.737792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e6518b137f4fb086720ac8747428d2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:28.740069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:28.760977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:28.801534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e6518b137f4fb086720ac8747428d2', 'request_id': 'req-34481602-e5b3-4803-a32a-2d004d525d0b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.808882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e6518b137f4fb086720ac8747428d2', 'request_id': 'req-34481602-e5b3-4803-a32a-2d004d525d0b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.818269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e6518b137f4fb086720ac8747428d2', 'request_id': 'req-34481602-e5b3-4803-a32a-2d004d525d0b'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.833417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] The token cache is being invalidate because project 42e6518b137f4fb086720ac8747428d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:28.833417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.833976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.835927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1233/2469] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:28 2022] DELETE /identity/v3/projects/42e6518b137f4fb086720ac8747428d2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:28.840542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:28.842059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:28.856194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341bf1a-97ab-40cf-a274-836a2c6f5524 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:28.925736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:28.926540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:28.926913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:28.927354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] PATH_INFO: `/v3/users/ded937513d8d4e9a8321df84821b131b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:28.929122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded937513d8d4e9a8321df84821b131b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:28.932041 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:28.989058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded937513d8d4e9a8321df84821b131b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:28.989910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded937513d8d4e9a8321df84821b131b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.001577 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded937513d8d4e9a8321df84821b131b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.010426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded937513d8d4e9a8321df84821b131b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.029286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e2d0d02-8b6c-4242-b876-b9024c34b497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded937513d8d4e9a8321df84821b131b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.038077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1237/2470] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:28 2022] DELETE /identity/v3/users/ded937513d8d4e9a8321df84821b131b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:29.363339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:29.364876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:29.378797 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481602-e5b3-4803-a32a-2d004d525d0b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:29.450445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:29.451325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.451601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.451874 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] PATH_INFO: `/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.453065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a18ad0fcb14c938840b2a93f94ffde)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:29.455094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:29.515555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a18ad0fcb14c938840b2a93f94ffde', 'request_id': 'req-1b250895-eb50-4bbe-937a-3dad272576d5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.524744 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a18ad0fcb14c938840b2a93f94ffde', 'request_id': 'req-1b250895-eb50-4bbe-937a-3dad272576d5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.534215 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a18ad0fcb14c938840b2a93f94ffde', 'request_id': 'req-1b250895-eb50-4bbe-937a-3dad272576d5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.547707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] The token cache is being invalidate because project f1a18ad0fcb14c938840b2a93f94ffde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:29.547952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.548729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-1b250895-eb50-4bbe-937a-3dad272576d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:29.550995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1234/2471] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:29 2022] DELETE /identity/v3/projects/f1a18ad0fcb14c938840b2a93f94ffde => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:29.557237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.557799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.557799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.583179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:29.584646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:29.621074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:29.667403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1238/2472] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:29.670955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.671849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.671849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.698496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:29.701119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:29.720186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1235/2473] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:29.724447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:29.734668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eca4d-5d09-4f7d-8b27-92fa5d48b312 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:29.741151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:29.741824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.742220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.742994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.743193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:59:29.743487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:29.745252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:29.763262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1239/2474] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 12 23:59:29.764831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:29.775962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b3bc4-4c0a-4ebf-ad30-4450d5202de5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:29.781405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:29.782250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.782619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.782982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.783805 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:29.786307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:29.822731 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1236/2475] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:29 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:29.827384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:29.840743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d52eed-6ddf-42ad-930a-6dac28e2d841 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:29.927521 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:29.927772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:29.928161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:29.928526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:29.930043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:29.934246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:29.977445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1240/2476] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:29 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:29.986477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:29.994530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:29.999654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fbba6-a82c-46bc-bd7f-0730404632bd admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:30.007037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:30.008036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:30.008708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:30.009372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:30.010466 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:30.012286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:30.024255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1237/2477] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:30.029073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:30.039724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24615f85-1832-4530-bc80-6f38b8f28e44 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:30.046033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:30.046995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:30.047596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:30.048193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:30.049305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:30.052210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484cbb-5842-4fca-b067-d9315f62c932 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:30.081960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1241/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:30.083214 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:30.093658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661b77-1712-49bf-80e2-65b0a58a12de admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:30.100164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:30.101063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:30.101407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:30.101876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] PATH_INFO: `/v3/projects/0d4c43138bd3461fb6d8fa9f924f7e99/users/568cea020ffb4b3fa74215dad3c17eba/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:30.128285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4c43138bd3461fb6d8fa9f924f7e99, user_id=568cea020ffb4b3fa74215dad3c17eba, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:30.132138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:30.151849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1238/2479] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:30 2022] PUT /identity/v3/projects/0d4c43138bd3461fb6d8fa9f924f7e99/users/568cea020ffb4b3fa74215dad3c17eba/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:30.157319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff7c8e-c32e-4e49-af94-0df05ea133ee None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:30.157602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff7c8e-c32e-4e49-af94-0df05ea133ee None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:30.158147 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff7c8e-c32e-4e49-af94-0df05ea133ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:30.180879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b7ff7c8e-c32e-4e49-af94-0df05ea133ee None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:30.182148 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b7ff7c8e-c32e-4e49-af94-0df05ea133ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:30.283633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1242/2480] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:30.301552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:30.303735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:30.374989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b92d18-ebe6-4b55-ac27-a244b0fe6775 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:30.384914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:30.388258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:30.388258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:30.388258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:30.411367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:30.411367 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:30.418532 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1239/2481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:31.188711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d313-ae5d-4b42-ab95-0546beb0392b None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:31.189695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d313-ae5d-4b42-ab95-0546beb0392b None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:31.190974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d313-ae5d-4b42-ab95-0546beb0392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:31.229713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1724d313-ae5d-4b42-ab95-0546beb0392b None None] MFA Rules not processed for user `568cea020ffb4b3fa74215dad3c17eba`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:31.231097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1724d313-ae5d-4b42-ab95-0546beb0392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:31.277057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:31.378738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1243/2482] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:31.391537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:31.393823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:31.503554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c0d71-eac5-4201-9f2f-b6ca419546cf service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:31.509881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:31.510684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:31.511086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:31.511509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:31.521749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:31.523343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:31.533579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1240/2483] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:31.954595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6abea-b723-4f8f-a3c1-675944bd5c7a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:31.954595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6abea-b723-4f8f-a3c1-675944bd5c7a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:31.954595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6abea-b723-4f8f-a3c1-675944bd5c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:31.984133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c0d6abea-b723-4f8f-a3c1-675944bd5c7a None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:31.986136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c0d6abea-b723-4f8f-a3c1-675944bd5c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:32.060424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1244/2484] 10.222.0.26 () {58 vars in 967 bytes} [Mon Sep 12 23:59:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:32.076909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:32.078377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:32.172913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487dfbb-0a34-4dd3-9221-b38da1860e17 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:32.178905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:32.179563 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:32.179851 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:32.180113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:32.189252 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:32.190864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:32.199931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1241/2485] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Sep 12 23:59:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 12 23:59:32.959004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:32.959678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:32.959678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:32.981453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] MFA Rules not processed for user `568cea020ffb4b3fa74215dad3c17eba`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:32.982895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:32.990975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1245/2486] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:42.196366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:42.975409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=42 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:47.052339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:47.054133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:47.071151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f21b4f-5bba-4175-a553-c07c5ce4c227 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:47.077387 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:47.078179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:47.078522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:47.078868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:47.081339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:47.097000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:47.098642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:47.110842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1242/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:51.807955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:51.810608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:51.830237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03368cc-c8bf-4d79-aa03-c0132f271bd0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:51.835014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:51.835711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:51.836031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:51.836255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] PATH_INFO: `/v3/users/a8cbfc74644f4d4594a70b833ad7002c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:51.837482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8cbfc74644f4d4594a70b833ad7002c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:51.839463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:51.897502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8cbfc74644f4d4594a70b833ad7002c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:51.898311 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8cbfc74644f4d4594a70b833ad7002c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:51.906824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8cbfc74644f4d4594a70b833ad7002c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:51.918076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8cbfc74644f4d4594a70b833ad7002c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:51.933440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-cba19c97-78b8-4723-91b7-0bafb1dd0816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8cbfc74644f4d4594a70b833ad7002c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:51.941355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1246/2488] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Sep 12 23:59:51 2022] DELETE /identity/v3/users/a8cbfc74644f4d4594a70b833ad7002c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:52.246030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.261543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19005bb5-cbeb-4b02-8ef6-934923e29ed5 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.351974 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.352775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.353180 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.353559 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] PATH_INFO: `/v3/projects/6c681f1232de4a5fbfc728c230f80624` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.355678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c681f1232de4a5fbfc728c230f80624)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.359519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.432517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c681f1232de4a5fbfc728c230f80624', 'request_id': 'req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:52.440353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c681f1232de4a5fbfc728c230f80624', 'request_id': 'req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:52.450930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c681f1232de4a5fbfc728c230f80624', 'request_id': 'req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:52.465644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] The token cache is being invalidate because project 6c681f1232de4a5fbfc728c230f80624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 12 23:59:52.466064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:52.466850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-7482e74d-cd14-4c28-8cfb-e527abc2c9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 12 23:59:52.468541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1243/2489] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Sep 12 23:59:52 2022] DELETE /identity/v3/projects/6c681f1232de4a5fbfc728c230f80624 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 12 23:59:52.474055 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.474335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.474608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.498379 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:52.499921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:52.573942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1247/2490] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:52.582540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.582819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.583095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.604781 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:52.606053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:52.614675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1244/2491] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:52.619280 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.632304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3153e-a1a8-4a66-bc67-10cc882a5535 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.637658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.638300 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.638581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.638853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.639571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 12 23:59:52.639878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.641621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf21994-1859-4541-90b9-9ad1c12ff85f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.655716 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1248/2492] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Sep 12 23:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 12 23:59:52.660662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.672857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c3715-9e16-4fc8-a79a-15596b2a3093 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.678681 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.679429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.679799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.680087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.680770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.682863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.684701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.685096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.685500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.718072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] MFA Rules not processed for user `568cea020ffb4b3fa74215dad3c17eba`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:52.719872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:52.733165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1245/2493] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:52 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:52.734523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.793880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:52.812181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1249/2494] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 12 23:59:52.813562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745a905-55cd-40b5-80d6-8ce35182a140 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.821038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.821679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.821959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.822362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.823507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.825594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.854433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1246/2495] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:52 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:52.859370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.866866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:52.873570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fae19-d4bf-4988-aedb-283a75a7d197 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.881294 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.882867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.883145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.883501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.884307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.885947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.898470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1250/2496] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:52.903793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.921278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87302a14-cc04-402d-a0b8-44a2faca6016 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.928938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.929894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.930340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.930779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.931980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:52.935127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d399ae2-b1d0-4fe9-8456-455177f0a982 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:52.959103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1247/2497] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 12 23:59:52.960540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:52.972615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3d0ab-8506-4235-bc81-59078606990b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:52.978013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:52.978722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:52.979209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:52.979520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] PATH_INFO: `/v3/projects/8e28ed6b6b1e49dfa5c0e4da218cddad/users/b3eb5f9e8e0741fd913665821b154d8b/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:52.998608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28ed6b6b1e49dfa5c0e4da218cddad, user_id=b3eb5f9e8e0741fd913665821b154d8b, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.001447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.020374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1251/2498] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:52 2022] PUT /identity/v3/projects/8e28ed6b6b1e49dfa5c0e4da218cddad/users/b3eb5f9e8e0741fd913665821b154d8b/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:53.032984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.033497 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.033912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.074141 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] MFA Rules not processed for user `b3eb5f9e8e0741fd913665821b154d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:53.076325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:53.175535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1248/2499] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:53.181184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.269948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c154c9-e1d1-4a93-a925-eb2231b03eab admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.275550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.276215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.276499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.276773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.277482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.279629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.321024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1252/2500] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Sep 12 23:59:53 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 12 23:59:53.326121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.339211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bea47-494c-4d0b-9f93-98f940d601d9 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.423828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.424485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.424761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.425025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.425705 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.427808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.463602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1249/2501] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Sep 12 23:59:53 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 12 23:59:53.465137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.479070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446346b1-5780-43ec-aecd-08081a56090e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.485064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.485914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.486289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.486710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.487517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.489361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.507355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1253/2502] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:53.513673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.527983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31797fcd-1d79-4cb9-8803-f6df63f3dc50 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.535922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.537127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.537701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.538188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] PATH_INFO: `/v3/projects/ff8c82ea92cd4833aa614b449ce40c45/users/949de650f6234e569ddd078115e256f9/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.560460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8c82ea92cd4833aa614b449ce40c45, user_id=949de650f6234e569ddd078115e256f9, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.563999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.582914 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1250/2503] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:53 2022] PUT /identity/v3/projects/ff8c82ea92cd4833aa614b449ce40c45/users/949de650f6234e569ddd078115e256f9/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:53.588136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.604630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677ca5b-6631-4944-8f8a-58a828fbf422 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.693438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.694461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.694931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.695433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.696622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.699704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3bbba-d19b-403a-ab30-99cd1e667acf admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.719639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1254/2504] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Sep 12 23:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 12 23:59:53.725054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:53.743878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f211b-d95c-48e7-b2ae-0a58b5460d7a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:53.751309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:53.752401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.752727 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.753120 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] PATH_INFO: `/v3/projects/ff8c82ea92cd4833aa614b449ce40c45/users/949de650f6234e569ddd078115e256f9/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.756177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8c82ea92cd4833aa614b449ce40c45, user_id=949de650f6234e569ddd078115e256f9, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:53.761784 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:53.780672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1251/2505] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Sep 12 23:59:53 2022] PUT /identity/v3/projects/ff8c82ea92cd4833aa614b449ce40c45/users/949de650f6234e569ddd078115e256f9/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 12 23:59:53.794333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:53.794774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:53.795259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:53.839452 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] MFA Rules not processed for user `949de650f6234e569ddd078115e256f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 12 23:59:53.841097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:53.893183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:53.978236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1255/2506] 10.222.0.26 () {58 vars in 958 bytes} [Mon Sep 12 23:59:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 12 23:59:53.997825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:54.001188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 12 23:59:54.020980 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 12 23:59:54.091849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003795ff-b38c-4bd1-b5c6-deaedd6bb931 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:54.097955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:54.098821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:54.099192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:54.099736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:54.186776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:54.189449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:54.202856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1252/2507] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 12 23:59:59.101887 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 12 23:59:59.112694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77b6f-6e3b-4412-bf32-6046029c5969 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 12 23:59:59.116427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 12 23:59:59.117051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 12 23:59:59.117322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 12 23:59:59.117627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 12 23:59:59.127708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 12 23:59:59.129145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312aa09-c031-4e62-994e-ba7f85891aeb service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 12 23:59:59.141671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1256/2508] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Sep 12 23:59:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:04.196719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=44 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:09.134918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=22 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:22.487001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:22.489211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:22.589505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c723-93d9-4aa9-8fa9-df1d074a11e2 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:22.596014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:22.596626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:22.596902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:22.597171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:22.599072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:22.665072 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:22.666694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:22.677751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1253/2509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:25.732931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:25.733447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:25.733773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:25.759435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:25.761130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:25.851273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1257/2510] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 13 00:00:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:25.884385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:25.886320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:25.971118 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d572c6-7885-46ac-a7ba-9b989db0817d service nova] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:25.977281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:25.978975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:25.979556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:25.980439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:25.998238 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:26.002308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf349-0187-4309-bca2-95070a2ddbeb service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:26.017275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1254/2511] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:00:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 00:00:26.950146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:27.055496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5d07b-5312-43d1-b238-f9af849aea27 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:27.067911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:27.070084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:27.070722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:27.071273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] PATH_INFO: `/v3/users/b3eb5f9e8e0741fd913665821b154d8b` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:27.073449 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3eb5f9e8e0741fd913665821b154d8b)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:27.076036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:27.164240 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3eb5f9e8e0741fd913665821b154d8b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:27.166984 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3eb5f9e8e0741fd913665821b154d8b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:27.181164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3eb5f9e8e0741fd913665821b154d8b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:27.204856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3eb5f9e8e0741fd913665821b154d8b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:27.241324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3eb5f9e8e0741fd913665821b154d8b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:27.274360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:27.275085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:27.275293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:27.281203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1258/2512] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:26 2022] DELETE /identity/v3/users/b3eb5f9e8e0741fd913665821b154d8b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:27.308706 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:27.310506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:27.337449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:27.401320 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1255/2513] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:27.412203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:27.413715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:27.457718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:27.532836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9f96-4133-4adb-9e83-0078042e1ed5 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:27.541943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:27.543934 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:27.545091 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:27.546383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:27.572672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:27.578010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:27.598929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1259/2514] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:28.056941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:28.058861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:28.083957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f2bf7-4083-463a-b986-bd92754b93c6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:28.089368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:28.090498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:28.091049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:28.091603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] PATH_INFO: `/v3/projects/8e28ed6b6b1e49dfa5c0e4da218cddad` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:28.111700 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e28ed6b6b1e49dfa5c0e4da218cddad)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:28.115119 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:28.220813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e28ed6b6b1e49dfa5c0e4da218cddad', 'request_id': 'req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.230765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e28ed6b6b1e49dfa5c0e4da218cddad', 'request_id': 'req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.252733 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e28ed6b6b1e49dfa5c0e4da218cddad', 'request_id': 'req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.270304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] The token cache is being invalidate because project 8e28ed6b6b1e49dfa5c0e4da218cddad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:28.270533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.271429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.273684 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1256/2515] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:28 2022] DELETE /identity/v3/projects/8e28ed6b6b1e49dfa5c0e4da218cddad => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:28.280516 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:28.305346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416441dd-2a20-4b44-b5f5-52910101223e service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:28.503381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:28.504290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:28.504570 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:28.504829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] PATH_INFO: `/v3/users/949de650f6234e569ddd078115e256f9` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:28.506853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=949de650f6234e569ddd078115e256f9)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:28.509409 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:28.622307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949de650f6234e569ddd078115e256f9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.623519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949de650f6234e569ddd078115e256f9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.636602 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949de650f6234e569ddd078115e256f9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.649477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949de650f6234e569ddd078115e256f9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.668406 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-a9108418-4f79-4db0-be9d-63a43619c67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949de650f6234e569ddd078115e256f9'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:28.680103 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1260/2516] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:28 2022] DELETE /identity/v3/users/949de650f6234e569ddd078115e256f9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:29.331970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:29.333459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:29.347924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f92dd-d4c0-4f94-ae30-3094bd581a67 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:29.457971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:29.458641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:29.458911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:29.459182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] PATH_INFO: `/v3/projects/ff8c82ea92cd4833aa614b449ce40c45` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:29.460635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8c82ea92cd4833aa614b449ce40c45)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:29.462636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:29.528386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8c82ea92cd4833aa614b449ce40c45', 'request_id': 'req-9ed39434-147a-4d29-ba78-e04fe82c0c33'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:29.531457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:29.543894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8c82ea92cd4833aa614b449ce40c45', 'request_id': 'req-9ed39434-147a-4d29-ba78-e04fe82c0c33'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:29.558611 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8c82ea92cd4833aa614b449ce40c45', 'request_id': 'req-9ed39434-147a-4d29-ba78-e04fe82c0c33'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:29.605759 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] The token cache is being invalidate because project ff8c82ea92cd4833aa614b449ce40c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:29.606055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:29.606682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9ed39434-147a-4d29-ba78-e04fe82c0c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:29.608728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1257/2517] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:29 2022] DELETE /identity/v3/projects/ff8c82ea92cd4833aa614b449ce40c45 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:29.614853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:29.615477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:29.615477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:29.645181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:29.646915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:29.714306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:29.748281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1261/2518] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:29.750606 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:29.750897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:29.751262 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:29.816982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:29.822172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:29.844274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1258/2519] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:29.849410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:29.895286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5983-43be-4f6c-b6a6-43054c39bf02 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:29.901263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:29.902277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:29.902554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:29.906218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:29.906218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:29.906218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:29.906218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:29.935696 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1262/2520] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 00:00:29.937574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:29.969556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6a138-eb39-49bd-9449-83f7857abc1c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:29.991175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:29.993418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:29.995929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:29.997336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:29.998222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.006085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.104131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1259/2521] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:29 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:30.107544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.140520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f45d0-acb2-4be9-b081-b5cdd089b37a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:30.287486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:30.287486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.287486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.287486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.288069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.288069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.292807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1263/2522] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:30 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:30.315398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.339453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95587-3441-4a3d-96ce-4f774a375cfa admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:30.363184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:30.364053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.364396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.364787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.365957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.373888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.401608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1260/2523] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:30.407616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.429905 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062c70-0a19-4957-a554-c417158971a9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:30.435748 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:30.436375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.436652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.436918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.455633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.455633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203aac9-d5ad-44d4-a6ea-e1e9ae8914f0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.461755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1264/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:30.466048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.478377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab31e-964d-4b81-b98b-211f86945761 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:30.484016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:30.484294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.484553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.484820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] PATH_INFO: `/v3/projects/840ee35e97544b529224cf299b2622bf/users/c9be064c0e7f4db1882cda23f75ef3b1/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.525845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=840ee35e97544b529224cf299b2622bf, user_id=c9be064c0e7f4db1882cda23f75ef3b1, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.531742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.560801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1261/2525] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:30 2022] PUT /identity/v3/projects/840ee35e97544b529224cf299b2622bf/users/c9be064c0e7f4db1882cda23f75ef3b1/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:30.579909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.579909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.579909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.627018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] MFA Rules not processed for user `c9be064c0e7f4db1882cda23f75ef3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:30.628407 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:30.734472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1265/2526] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:30.739345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.870859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea33eb-9c79-4a17-8931-15dac3af72ee admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:30.877294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:30.878055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:30.878396 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:30.878767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:30.879565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:30.881695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:30.923422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1262/2527] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:30 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:30.924172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:30.945296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd746c0d-2851-4565-bd6a-24a6a863ac6e None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.068031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:31.110221 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.112020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.112802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.114463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.115814 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.119328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.173701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1266/2528] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:30 2022] POST /identity/v3/users => generated 290 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:31.178103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:31.195308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea5ba4-b311-411c-95fc-e2af385fc6e3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.201108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.202045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.202522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.202954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.203818 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.207291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.224375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1263/2529] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:31.225640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:31.243681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f923b-8d37-428d-a2ed-8c9cea356629 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.252935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.254252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.254709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.255005 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] PATH_INFO: `/v3/projects/fa6e3d82314149c6b98efb071c27aeff/users/760010a46ab8407b9734da5e1183a9fe/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.287252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6e3d82314149c6b98efb071c27aeff, user_id=760010a46ab8407b9734da5e1183a9fe, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.290248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.319267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1267/2530] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:31 2022] PUT /identity/v3/projects/fa6e3d82314149c6b98efb071c27aeff/users/760010a46ab8407b9734da5e1183a9fe/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:31.321404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:31.335551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e9362-8bda-45f3-b8ba-0d423f26acee admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.426608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:31.447432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.449598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.450374 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.450949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.452444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.456327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d66a87-1706-46aa-8ef3-17f3d6755da9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.479422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1264/2531] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:31.483755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:31.501735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28794fb5-bcbe-4dbe-aea9-4bdd726e0cc6 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.513865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.515648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.517968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.517968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] PATH_INFO: `/v3/projects/fa6e3d82314149c6b98efb071c27aeff/users/760010a46ab8407b9734da5e1183a9fe/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.525401 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6e3d82314149c6b98efb071c27aeff, user_id=760010a46ab8407b9734da5e1183a9fe, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.536546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.578852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.579774 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.580476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.584888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1268/2532] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:31 2022] PUT /identity/v3/projects/fa6e3d82314149c6b98efb071c27aeff/users/760010a46ab8407b9734da5e1183a9fe/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:31.625429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] MFA Rules not processed for user `760010a46ab8407b9734da5e1183a9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:31.626834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:31.761926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1265/2533] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:31.790776 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:31.793021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:31.898443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fd2d-aa10-495a-9ec6-154498ccac2a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:31.906536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:31.907383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:31.907730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:31.908072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:31.923341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:31.925446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:31.941094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1269/2534] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:32.004363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:32.023298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993680e-8b5c-4002-bebf-d1f62695b541 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:32.030092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:32.031315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:32.031828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:32.032493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:32.184678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:32.188696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:32.222413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1266/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:34.108506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:34.110002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:34.184314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c587a0-cedf-4ee3-a4d3-5a8ad22b18e1 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:34.190595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:34.191850 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:34.192475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:34.193051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:34.195273 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:34.274847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:34.316260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:34.322111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4fe3-db9a-46ee-aa9b-d49d0c615e83 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:34.338712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1270/2536] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:36.799936 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:36.848610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589bbcf-120f-4f84-83cd-5167feb71710 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:36.874016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:36.880444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:36.881683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:36.887764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] PATH_INFO: `/v3/users/c9be064c0e7f4db1882cda23f75ef3b1` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:36.892628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9be064c0e7f4db1882cda23f75ef3b1)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:36.896404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:37.062739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9be064c0e7f4db1882cda23f75ef3b1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:37.063594 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9be064c0e7f4db1882cda23f75ef3b1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:37.064260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9be064c0e7f4db1882cda23f75ef3b1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:37.082724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9be064c0e7f4db1882cda23f75ef3b1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:37.094840 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:37.105502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9be064c0e7f4db1882cda23f75ef3b1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:37.128007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1267/2537] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:36 2022] DELETE /identity/v3/users/c9be064c0e7f4db1882cda23f75ef3b1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:37.129740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:37.130269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:37.130269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:37.161359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:37.163355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:37.286403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1271/2538] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:37.299777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:37.302624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:37.403244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab456af-45fd-4f1f-a25e-d006c39da61e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:37.410212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:37.410623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:37.411040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:37.411353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:37.432465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:37.435011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:37.469918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1268/2539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:38.018708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:38.021512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:38.036796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7b0b9-1038-4132-9192-11ff5d361b06 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:38.047308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:38.048701 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:38.048989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:38.049170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] PATH_INFO: `/v3/projects/840ee35e97544b529224cf299b2622bf` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:38.060583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840ee35e97544b529224cf299b2622bf)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:38.063043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:38.224274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840ee35e97544b529224cf299b2622bf', 'request_id': 'req-f5398379-1fe1-47c1-ae1a-5e9c1b454257'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.235734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840ee35e97544b529224cf299b2622bf', 'request_id': 'req-f5398379-1fe1-47c1-ae1a-5e9c1b454257'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.244430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840ee35e97544b529224cf299b2622bf', 'request_id': 'req-f5398379-1fe1-47c1-ae1a-5e9c1b454257'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.266644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] The token cache is being invalidate because project 840ee35e97544b529224cf299b2622bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:38.266936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.269375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.271421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1272/2540] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:38 2022] DELETE /identity/v3/projects/840ee35e97544b529224cf299b2622bf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:38.275919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:38.292200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757910-3577-48fb-a77c-101f19d68272 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:38.380626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:38.382024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:38.382932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:38.383558 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] PATH_INFO: `/v3/users/760010a46ab8407b9734da5e1183a9fe` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:38.387598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=760010a46ab8407b9734da5e1183a9fe)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:38.393649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:38.465359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760010a46ab8407b9734da5e1183a9fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.466096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760010a46ab8407b9734da5e1183a9fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.466927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760010a46ab8407b9734da5e1183a9fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.480036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760010a46ab8407b9734da5e1183a9fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.501373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-619f85e8-2f8b-4394-8ffa-fb1807afca7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760010a46ab8407b9734da5e1183a9fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:38.536368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1269/2541] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:38 2022] DELETE /identity/v3/users/760010a46ab8407b9734da5e1183a9fe => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:39.340112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:39.342526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:39.373007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5398379-1fe1-47c1-ae1a-5e9c1b454257 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:39.500084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:39.501455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:39.502458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:39.502956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] PATH_INFO: `/v3/projects/fa6e3d82314149c6b98efb071c27aeff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:39.509553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6e3d82314149c6b98efb071c27aeff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:39.514809 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:39.542396 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:39.619388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6e3d82314149c6b98efb071c27aeff', 'request_id': 'req-caf71651-53db-40a7-8f99-a7371c2bcbe2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:39.628507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6e3d82314149c6b98efb071c27aeff', 'request_id': 'req-caf71651-53db-40a7-8f99-a7371c2bcbe2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:39.640013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6e3d82314149c6b98efb071c27aeff', 'request_id': 'req-caf71651-53db-40a7-8f99-a7371c2bcbe2'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:39.674547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] The token cache is being invalidate because project fa6e3d82314149c6b98efb071c27aeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:39.675664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:39.681904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-caf71651-53db-40a7-8f99-a7371c2bcbe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:39.685002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1273/2542] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:39 2022] DELETE /identity/v3/projects/fa6e3d82314149c6b98efb071c27aeff => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:39.692170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:39.692827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:39.693150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:39.707168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:39.729500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:39.741654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:39.932200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1270/2543] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:39.942419 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:39.943189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:39.944019 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:39.982510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:39.984930 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.000226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1274/2544] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:40.004949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.023300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e121-d673-4652-bf2f-1fa34d5149c7 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.038173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.038834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.039083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.039384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.040058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:40.040358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.042492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.069822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.072124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.073890 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1271/2545] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:00:40.082844 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.111476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c68370-72fd-4fbc-bc4d-c0dd25712d8d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.123460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.135556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.144209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.144538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.145512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.148123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.224837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1272/2546] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:40 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:40.233807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.238600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.268181 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7e343-68dc-4d7e-800f-5055840f20f3 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.374196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.376283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.376979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.377501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.380305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.397894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.400099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.414993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1275/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:40.416472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.427642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008d65f-9a10-4b0e-a027-9599f5ae60d7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.433851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66430-0e76-4e08-b4e2-a62d25464f3c service placement] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.434728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.442884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.443127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.443713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.443713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] PATH_INFO: `/v3/users/296f02c73fc548969bf60df4f8f372b0` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.444759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.444968 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296f02c73fc548969bf60df4f8f372b0)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.445124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.445507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.446600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.447363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.448542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.481204 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1276/2548] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:40 2022] POST /identity/v3/users => generated 292 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:40.489856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.505342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1594-92f7-4e66-bad7-783b922fe3ee admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.512480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.513878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.514309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.514627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.515949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.518637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.522548 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:40.536432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296f02c73fc548969bf60df4f8f372b0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:40.538058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296f02c73fc548969bf60df4f8f372b0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:40.538770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296f02c73fc548969bf60df4f8f372b0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:40.542483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1277/2549] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:40.548774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.550056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296f02c73fc548969bf60df4f8f372b0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:40.551916 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.566565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:40.569959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296f02c73fc548969bf60df4f8f372b0'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:40.575937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f5288-80d5-4094-bdf9-89034871e1a5 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.586803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.586803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.586803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.586803 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.587544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.589590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009660b-11ca-48af-98b0-59ec4121365d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.601586 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1273/2550] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:40 2022] DELETE /identity/v3/users/296f02c73fc548969bf60df4f8f372b0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:40.604325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1278/2551] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:40.609655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.703008 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33419c54-7d03-4ebf-8165-6ba607445b95 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:40.709975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:40.710718 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.711168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.711492 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] PATH_INFO: `/v3/projects/c644d6bfa4684beea1f62eb64109e6b2/users/305b6f6039f64462a26febd81b8a9e8f/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.744240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c644d6bfa4684beea1f62eb64109e6b2, user_id=305b6f6039f64462a26febd81b8a9e8f, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:40.751535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:40.770884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1274/2552] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:40 2022] PUT /identity/v3/projects/c644d6bfa4684beea1f62eb64109e6b2/users/305b6f6039f64462a26febd81b8a9e8f/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:40.785127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:40.786259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:40.786442 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:40.829928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] MFA Rules not processed for user `305b6f6039f64462a26febd81b8a9e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:40.831525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.917063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:40.920633 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:40.979522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1279/2553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:40.984364 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.039993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee16a9-5f8f-492a-8042-0924f771914c admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.066566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.074993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.074993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.074993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] PATH_INFO: `/v3/projects/6b2f385a651c408bb9d32b5c4e1ab6ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.074993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2f385a651c408bb9d32b5c4e1ab6ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.079891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.120468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42021cc9-ce27-407e-84d1-b27ded0eb6af None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.131583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.132808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.133168 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.133566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.134817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.137480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.161316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2f385a651c408bb9d32b5c4e1ab6ff', 'request_id': 'req-279ad6d6-1269-47ee-92a4-517dad991ddd'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:41.171506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2f385a651c408bb9d32b5c4e1ab6ff', 'request_id': 'req-279ad6d6-1269-47ee-92a4-517dad991ddd'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:41.194429 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1280/2554] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:40 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:41.198033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.206481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2f385a651c408bb9d32b5c4e1ab6ff', 'request_id': 'req-279ad6d6-1269-47ee-92a4-517dad991ddd'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:41.230146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] The token cache is being invalidate because project 6b2f385a651c408bb9d32b5c4e1ab6ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:41.231248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:41.232153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-279ad6d6-1269-47ee-92a4-517dad991ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:41.234056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1275/2555] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:40 2022] DELETE /identity/v3/projects/6b2f385a651c408bb9d32b5c4e1ab6ff => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:41.239644 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f4e1-f9ba-4265-ae1d-007ec39c9cd6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.240129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f4e1-f9ba-4265-ae1d-007ec39c9cd6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.240673 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f4e1-f9ba-4265-ae1d-007ec39c9cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.250772 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:41.268447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-55f4f4e1-f9ba-4265-ae1d-007ec39c9cd6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:41.270258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-55f4f4e1-f9ba-4265-ae1d-007ec39c9cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:41.304632 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442ac3-81c2-4ff7-a914-c0a4f13ca541 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.310359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.311362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.311848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.312615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.313524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.316978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.330736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:41.354942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1281/2556] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:00:41.356008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.360177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:41.364469 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1276/2557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:00:41.375081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.376012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25086-a866-4ed7-b1be-9f2aa1517f86 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.377756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.378129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.382735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.385084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.385713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.386305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.387486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.389400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.407272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1282/2558] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:41.411493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.421444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:41.424660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:41.432037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe00a8-b21a-4495-a109-522cb97165d7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.440329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.440940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.441215 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.441489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] PATH_INFO: `/v3/projects/7ca58d540ec342b59c45239752e68914/users/562573333c094b1eae8f567491613a85/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.455419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1277/2559] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:41.456414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.467127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fd8c8-4d4d-4e33-9063-5585670e7844 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.468254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca58d540ec342b59c45239752e68914, user_id=562573333c094b1eae8f567491613a85, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.472682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.473425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.473556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.473807 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.474201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.474957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:41.475301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.477892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.493849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1278/2560] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:00:41.498800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.502243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1283/2561] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:41 2022] PUT /identity/v3/projects/7ca58d540ec342b59c45239752e68914/users/562573333c094b1eae8f567491613a85/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:41.506313 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.515109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc829f9-313a-4963-ad7b-ce05fd704ad2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.602314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599ac7-74b2-4030-9a13-456d9c97fb5e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.610164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.611015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.611159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.611435 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.612032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.613655 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.622842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.623688 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.624392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.624392 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.625548 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.627395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.640404 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1284/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:41.646583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.671635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47825fbc-ba3a-4a3e-b06f-3a2484eb24cd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.673407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1279/2563] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:00:41.674359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.777662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291185b7-e83a-4fb4-a1de-d11deca5e786 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.784598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.785886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.786363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.786819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.787702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.790155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.800205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.800796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.801077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.801349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] PATH_INFO: `/v3/projects/7ca58d540ec342b59c45239752e68914/users/562573333c094b1eae8f567491613a85/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.804335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca58d540ec342b59c45239752e68914, user_id=562573333c094b1eae8f567491613a85, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.807111 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60253-bae6-4a03-a14a-c428485eb1e9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.816596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:41.824824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1280/2565] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:41.831353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1285/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:41 2022] PUT /identity/v3/projects/7ca58d540ec342b59c45239752e68914/users/562573333c094b1eae8f567491613a85/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:41.837075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.846983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.847579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.847867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.851338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9a45-dd21-4fd6-bb43-741b75b7dd65 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:41.881500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] MFA Rules not processed for user `562573333c094b1eae8f567491613a85`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:41.882843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:41.932206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:41.944055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:41.946021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:41.946590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:41.947047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:41.947972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:41.950226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:41.963383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1281/2566] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:41.969911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:41.994537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500f82e-8740-4c18-9b07-1fbc62d73320 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.000833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.001728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.002002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.002501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.003264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:42.007547 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1286/2567] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:42.013965 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:42.024155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:42.025856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:42.030948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1282/2568] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:42.035850 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:42.051057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c9969-0ffc-47b8-b2ec-da7b547017ea admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.056838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.058667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.059042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.059418 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] PATH_INFO: `/v3/projects/c3f8028d6272470f99135b302b72bae7/users/8463f36f6b194e67aa1652f68c698a47/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.084524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f8028d6272470f99135b302b72bae7, user_id=8463f36f6b194e67aa1652f68c698a47, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:42.089075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2fdd3-4d03-4972-b14c-2c14a64e527b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:42.116034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb610c4a-23d4-4d30-bd40-af455c530086 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.125193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1283/2569] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:42 2022] PUT /identity/v3/projects/c3f8028d6272470f99135b302b72bae7/users/8463f36f6b194e67aa1652f68c698a47/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:42.128517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.128785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.129108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.195298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] MFA Rules not processed for user `8463f36f6b194e67aa1652f68c698a47`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:42.196952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:42.212646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.213715 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.213998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.214261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.306813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1284/2570] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:00:42.316653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:42.318118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:42.322908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:42.326266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:42.330098 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1287/2571] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:42.418368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:42.460297 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:42.490733 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad59adb-eec6-47b1-8852-efb8fc9da21c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.504356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.505455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.505897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.506168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.518910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:42.520626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df63d52-975b-4e37-b895-943aab10fce6 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:42.528576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:42.538019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1285/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:42.562048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b91c16-3582-4ace-a67e-2264d81cc01b service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.569754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.570796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.571411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.571411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] PATH_INFO: `/v3/users/305b6f6039f64462a26febd81b8a9e8f` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.573059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305b6f6039f64462a26febd81b8a9e8f)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:42.577093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:42.664683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305b6f6039f64462a26febd81b8a9e8f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:42.666233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305b6f6039f64462a26febd81b8a9e8f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:42.683063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305b6f6039f64462a26febd81b8a9e8f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:42.704143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305b6f6039f64462a26febd81b8a9e8f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:42.726420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305b6f6039f64462a26febd81b8a9e8f'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:42.744445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1288/2573] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:42 2022] DELETE /identity/v3/users/305b6f6039f64462a26febd81b8a9e8f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:42.746738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.746738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.746982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.769831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:42.771073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:42.871352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1286/2574] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:00:42.879173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:42.881062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:42.978490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b07ddb-73eb-472f-9f79-d79c367c6546 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:42.983976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:42.984998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:42.985836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:42.985836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:42.998566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:43.000250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:43.013706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1289/2575] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:43.496153 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:43.497869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:43.532575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aa965-0efb-4c4e-b13d-75939cf5e0bb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:43.541590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:43.542255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:43.542550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:43.542894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] PATH_INFO: `/v3/projects/c644d6bfa4684beea1f62eb64109e6b2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:43.543990 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c644d6bfa4684beea1f62eb64109e6b2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:43.545959 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:43.628616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c644d6bfa4684beea1f62eb64109e6b2', 'request_id': 'req-e2a7c267-8a03-4f17-afb4-4155b9682d18'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.638378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c644d6bfa4684beea1f62eb64109e6b2', 'request_id': 'req-e2a7c267-8a03-4f17-afb4-4155b9682d18'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.649536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c644d6bfa4684beea1f62eb64109e6b2', 'request_id': 'req-e2a7c267-8a03-4f17-afb4-4155b9682d18'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.667512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] The token cache is being invalidate because project c644d6bfa4684beea1f62eb64109e6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:43.667982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.668515 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.670761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1287/2576] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:43 2022] DELETE /identity/v3/projects/c644d6bfa4684beea1f62eb64109e6b2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:43.675370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:43.687205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd17fb-7bdd-41d1-b6ce-c5919264f1c1 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:43.780139 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:43.780970 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:43.781260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:43.781571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] PATH_INFO: `/v3/users/562573333c094b1eae8f567491613a85` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:43.783083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562573333c094b1eae8f567491613a85)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:43.785965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:43.788538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:43.860995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562573333c094b1eae8f567491613a85'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.862722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562573333c094b1eae8f567491613a85'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.882338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562573333c094b1eae8f567491613a85'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.895031 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562573333c094b1eae8f567491613a85'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.914478 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-36b97968-86dd-4054-a96f-fc86db9c3fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562573333c094b1eae8f567491613a85'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:43.921730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1290/2577] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:43 2022] DELETE /identity/v3/users/562573333c094b1eae8f567491613a85 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:44.630218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:44.632110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:44.661654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7c267-8a03-4f17-afb4-4155b9682d18 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:44.793583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:44.801995 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:44.802687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:44.803663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:44.804192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] PATH_INFO: `/v3/projects/7ca58d540ec342b59c45239752e68914` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:44.807028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca58d540ec342b59c45239752e68914)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:44.813579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:44.973029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca58d540ec342b59c45239752e68914', 'request_id': 'req-feb1003a-a142-4c90-b7e8-c360da068c83'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:44.982762 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca58d540ec342b59c45239752e68914', 'request_id': 'req-feb1003a-a142-4c90-b7e8-c360da068c83'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:44.994753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca58d540ec342b59c45239752e68914', 'request_id': 'req-feb1003a-a142-4c90-b7e8-c360da068c83'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:45.012882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] The token cache is being invalidate because project 7ca58d540ec342b59c45239752e68914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:45.013282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:45.014446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-feb1003a-a142-4c90-b7e8-c360da068c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:45.021226 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1288/2578] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:44 2022] DELETE /identity/v3/projects/7ca58d540ec342b59c45239752e68914 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:45.024203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.027756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.028573 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.074582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:45.076021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:45.179566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1291/2579] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:45.188956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.188956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.188956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.212799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:45.214336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:45.231327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1289/2580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:45.233848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.246839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5f696-6135-49f2-982f-fa15e373ddd0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.254704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.255519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.255972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.256578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.257580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:45.257908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.259888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.283145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1292/2581] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:00:45.286009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.313921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10304a-ff55-456c-bc60-4eae1ca8a2ee None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.327358 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.328464 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.329122 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.329465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.330809 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.348427 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.414841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.422975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1290/2582] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:45 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:45.430191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e27952-bb7d-4a67-8c22-802f5fd9f932 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.539092 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.539946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.540225 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.540550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.541246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.543485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.562487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:45.585523 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.591536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1293/2583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:45 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:45.599432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef67f-47e3-409c-b1b4-4da16cccfa32 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.623938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.625158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.625672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.627636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.628883 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.632019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.654826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1291/2584] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:45.655559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.674661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786aba70-e8e1-4f4b-a980-9bcda860fba1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.680802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.684233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.684584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.684917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.685787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.688574 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36d39-5192-4e6c-850e-474ac945d48e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.718267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.730888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bead198-533e-4f6f-b6f8-45d71acd9f6a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:45.732024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1294/2585] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:00:45.739373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:45.740067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.740337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.740607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] PATH_INFO: `/v3/projects/09cc6ad304824962b47f1f2767b29579/users/68fa4bf415794157b08c4bd9ae644339/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.757625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cc6ad304824962b47f1f2767b29579, user_id=68fa4bf415794157b08c4bd9ae644339, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:45.760337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:45.777847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1292/2586] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:45 2022] PUT /identity/v3/projects/09cc6ad304824962b47f1f2767b29579/users/68fa4bf415794157b08c4bd9ae644339/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:45.799335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:45.799335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:45.799335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:45.832637 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] MFA Rules not processed for user `68fa4bf415794157b08c4bd9ae644339`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:45.833997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:45.921154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1295/2587] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:45.933769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:45.935498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:45.999844 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:46.001068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:46.035591 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c9c1-1a6f-4d72-89cb-4bf9e5f964c3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:46.039428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:46.040160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:46.041277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:46.041277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:46.059875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:46.068650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f520b1e-ab84-4508-9ed6-88757703ba13 service glance] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:46.080236 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1293/2588] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:46.113455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4da48-706e-4cf1-aa0d-52482962c5d0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:46.121702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:46.122468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:46.122986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:46.123292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] PATH_INFO: `/v3/users/8463f36f6b194e67aa1652f68c698a47` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:46.124731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8463f36f6b194e67aa1652f68c698a47)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:46.127257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:46.201488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8463f36f6b194e67aa1652f68c698a47'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:46.203386 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8463f36f6b194e67aa1652f68c698a47'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:46.213145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8463f36f6b194e67aa1652f68c698a47'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:46.223792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8463f36f6b194e67aa1652f68c698a47'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:46.256910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:46.265374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8463f36f6b194e67aa1652f68c698a47'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:46.274234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1296/2589] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:45 2022] DELETE /identity/v3/users/8463f36f6b194e67aa1652f68c698a47 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:46.279691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:46.280044 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:46.280288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:46.301669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:46.303077 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:46.329315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:46.409911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1294/2590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:46.421138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:46.422991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:46.517412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40fa5-e80a-43b4-8def-b0fd931bb963 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:46.525045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:46.526354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:46.526600 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:46.526927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:46.545778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:46.546783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fe80f-a7f2-4aac-bd37-fd612731bd32 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:46.558982 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1297/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:00:47.086265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.088720 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:47.103185 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badaf4a5-5be4-494a-9249-a1885544eba0 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.109506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.113972 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.114168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.114600 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] PATH_INFO: `/v3/projects/c3f8028d6272470f99135b302b72bae7` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.117045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f8028d6272470f99135b302b72bae7)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.121253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.201352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f8028d6272470f99135b302b72bae7', 'request_id': 'req-2ab80f08-957a-4ca7-bfe8-64b5b7223333'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:47.212651 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f8028d6272470f99135b302b72bae7', 'request_id': 'req-2ab80f08-957a-4ca7-bfe8-64b5b7223333'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:47.225152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f8028d6272470f99135b302b72bae7', 'request_id': 'req-2ab80f08-957a-4ca7-bfe8-64b5b7223333'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:47.255198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] The token cache is being invalidate because project c3f8028d6272470f99135b302b72bae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:47.255888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:47.257075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-2ab80f08-957a-4ca7-bfe8-64b5b7223333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:47.264657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1295/2592] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:47 2022] DELETE /identity/v3/projects/c3f8028d6272470f99135b302b72bae7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:47.267635 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.268623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.268623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.313909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:47.315315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:47.411194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1298/2593] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:47.420722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.421158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.421682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.448780 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:47.449801 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:47.459240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1296/2594] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:47.464341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.475346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a224833-5173-4969-a27e-a857fc7f6932 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.485306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.485980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.486415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.486691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.487395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:47.487702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.489369 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.503071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1299/2595] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:00:47.507917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.520921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b83d2a-96f4-472c-942c-f804f25c7f9a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.527163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.527955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.528386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.528717 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.529395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.532519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.564661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1297/2596] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:47 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:47.569616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.580765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf6fdf-af4a-4dfc-9f64-7e885997c729 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.657962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.658653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.658954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.659269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.659892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.661921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.699086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1300/2597] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:47 2022] POST /identity/v3/users => generated 303 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:47.703790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.724123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa027-e52d-4a01-b57b-4a887fa1ef08 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.731785 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.732896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.733603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.734156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.735787 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.739829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.772173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1298/2598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:00:47.773578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.786214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c80049-abbf-4e3a-b704-137013b4ab25 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.792994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.793826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.794237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.794583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.795354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.798308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f432410-58bc-46fa-a1e2-d6919840d155 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.816146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1301/2599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:47.820854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:47.832291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a16f0a-2537-4d02-acf1-7194cc2d6f3a admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:47.840126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:47.840932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.841305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.841645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] PATH_INFO: `/v3/projects/00814ba4277a4b61a512de1c21d2ec71/users/9ccc2116323b4e23af6d5d8cb2e4191c/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.861310 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=00814ba4277a4b61a512de1c21d2ec71, user_id=9ccc2116323b4e23af6d5d8cb2e4191c, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:47.864572 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:47.880255 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1299/2600] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:47 2022] PUT /identity/v3/projects/00814ba4277a4b61a512de1c21d2ec71/users/9ccc2116323b4e23af6d5d8cb2e4191c/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:47.886222 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca7c4-85e3-451c-9cbe-466860ac5ae4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:47.886510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca7c4-85e3-451c-9cbe-466860ac5ae4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:47.886782 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca7c4-85e3-451c-9cbe-466860ac5ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:47.895006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:47.921123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-626ca7c4-85e3-451c-9cbe-466860ac5ae4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:47.923490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-626ca7c4-85e3-451c-9cbe-466860ac5ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:48.071886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1302/2601] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:48.083535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:48.085650 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:48.162969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:48.170663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f2ab0-e1e5-4011-9b5e-2ebe06ffc5ea admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:48.175738 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:48.176414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:48.176701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:48.176973 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:48.187881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:48.189604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:48.205603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1300/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:48.648326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa609ec3-bb05-4ea8-8e77-018570be04b1 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:48.648895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa609ec3-bb05-4ea8-8e77-018570be04b1 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:48.648895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa609ec3-bb05-4ea8-8e77-018570be04b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:48.693472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-aa609ec3-bb05-4ea8-8e77-018570be04b1 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:48.695134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-aa609ec3-bb05-4ea8-8e77-018570be04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:48.795385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1303/2603] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 13 00:00:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:48.810656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:48.812253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:48.909695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42331c6-181a-44a4-90f1-adb66869dd4c service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:48.916445 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:48.917244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:48.917622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:48.918026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:48.930755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:48.933341 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:48.946903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1301/2604] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:00:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 00:00:49.067212 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.068743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.069381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.151344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] MFA Rules not processed for user `9ccc2116323b4e23af6d5d8cb2e4191c`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:49.153006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:49.299312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1304/2605] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:49 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:49.302811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.317325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e26ac7-e950-4450-8f06-f05fd3fe0f25 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.323754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.324630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.325192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.325646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.327117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.329422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.374824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1302/2606] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:49 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:49.381522 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.396281 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b7bf-539a-40d1-ac87-3d945ba3034d None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.457644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=100 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:49.495796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.496292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.496571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.496843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.497493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.499494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.529966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1305/2607] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:49 2022] POST /identity/v3/users => generated 308 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:49.539032 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.551946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7274a-c080-49bb-96c0-7d7c98133260 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.557661 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.558372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.558658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.558945 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.559808 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.561637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.575750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1303/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:49.580952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.592440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c9d5f-40f0-43b9-a9b7-1865a9456ad9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.598183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.599264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.599702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.600110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] PATH_INFO: `/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405/users/28e208b6eb7e49bba714e5ff8f9e6b5e/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.625118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9ddffaf0ad480ca93ec028a6cc5405, user_id=28e208b6eb7e49bba714e5ff8f9e6b5e, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.628321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.647299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1306/2609] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:49 2022] PUT /identity/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405/users/28e208b6eb7e49bba714e5ff8f9e6b5e/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:49.651205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.664223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fb01-0f1b-4ce5-b13f-e5f2bf769843 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.763330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.764287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.764775 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.765251 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.767426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.770514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0e841-5288-4457-83d0-172d97e5e171 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.786291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1304/2610] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:00:49.791539 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:49.803308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f19392-1568-4b5d-ab6a-4d17da80893e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:49.808598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:49.809365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:49.809787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:49.810234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] PATH_INFO: `/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405/users/28e208b6eb7e49bba714e5ff8f9e6b5e/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:49.812667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9ddffaf0ad480ca93ec028a6cc5405, user_id=28e208b6eb7e49bba714e5ff8f9e6b5e, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:49.816391 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:49.829504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1307/2611] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:49 2022] PUT /identity/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405/users/28e208b6eb7e49bba714e5ff8f9e6b5e/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:50.603401 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:50.604015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:50.604307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:50.645356 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] MFA Rules not processed for user `28e208b6eb7e49bba714e5ff8f9e6b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:50.647294 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:50.713068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:50.778522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1305/2612] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:52.299515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:52.301351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:52.395087 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f182-cb46-42a5-ac87-100bb6651a37 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:52.401067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:52.401781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:52.402014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:52.402259 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] PATH_INFO: `/v3/users/568cea020ffb4b3fa74215dad3c17eba` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:52.403940 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=568cea020ffb4b3fa74215dad3c17eba)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:52.405891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:52.490132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568cea020ffb4b3fa74215dad3c17eba'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:52.490864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568cea020ffb4b3fa74215dad3c17eba'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:52.500026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568cea020ffb4b3fa74215dad3c17eba'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:52.511665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568cea020ffb4b3fa74215dad3c17eba'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:52.526357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568cea020ffb4b3fa74215dad3c17eba'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:52.533610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1308/2613] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:52 2022] DELETE /identity/v3/users/568cea020ffb4b3fa74215dad3c17eba => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:52.948385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:52.965861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b773-2eb5-4888-a3fd-7778a7a93fa7 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.085911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.089951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:53.121909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.124168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.124168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.124642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] PATH_INFO: `/v3/projects/0d4c43138bd3461fb6d8fa9f924f7e99` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.126526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:53.128671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4c43138bd3461fb6d8fa9f924f7e99)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.133227 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.248381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4c43138bd3461fb6d8fa9f924f7e99', 'request_id': 'req-ab341db9-1a01-404f-9278-240b4f43e8e6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.256134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4c43138bd3461fb6d8fa9f924f7e99', 'request_id': 'req-ab341db9-1a01-404f-9278-240b4f43e8e6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.267412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4c43138bd3461fb6d8fa9f924f7e99', 'request_id': 'req-ab341db9-1a01-404f-9278-240b4f43e8e6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.292227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba498892-eb52-4009-9da7-a3fc4a7cc0cc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.316438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] The token cache is being invalidate because project 0d4c43138bd3461fb6d8fa9f924f7e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:53.320357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.322060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ab341db9-1a01-404f-9278-240b4f43e8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.327856 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1306/2614] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:52 2022] DELETE /identity/v3/projects/0d4c43138bd3461fb6d8fa9f924f7e99 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:53.334103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572056b2-93eb-4aaf-9784-212ede058f39 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.335184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572056b2-93eb-4aaf-9784-212ede058f39 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.335184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572056b2-93eb-4aaf-9784-212ede058f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.376564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.377906 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-572056b2-93eb-4aaf-9784-212ede058f39 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:53.379870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-572056b2-93eb-4aaf-9784-212ede058f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:53.384023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.384514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.384876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] PATH_INFO: `/v3/users/9ccc2116323b4e23af6d5d8cb2e4191c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.386482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ccc2116323b4e23af6d5d8cb2e4191c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.389064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.396595 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1307/2615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:00:53.408461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.408969 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.409338 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.446115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:53.448843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:53.460452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1308/2616] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:53.466200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.479473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa039bb-dcb3-485c-ae60-7aa9efcb1b36 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.485712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ccc2116323b4e23af6d5d8cb2e4191c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.487337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ccc2116323b4e23af6d5d8cb2e4191c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.488354 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.488991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.490750 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.491025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.491779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:53.492108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.493944 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.517565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1309/2617] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:00:53.520955 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ccc2116323b4e23af6d5d8cb2e4191c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.522016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.538775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ccc2116323b4e23af6d5d8cb2e4191c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.544127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:53.557485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ccc2116323b4e23af6d5d8cb2e4191c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:53.576932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb703-ffcd-4038-9061-3d12db6cc891 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.589219 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1309/2618] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:53 2022] DELETE /identity/v3/users/9ccc2116323b4e23af6d5d8cb2e4191c => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:53.602979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.603960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.604375 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.604755 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.605599 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.608758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.692634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1310/2619] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:53 2022] POST /identity/v3/projects => generated 399 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:53.696865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.752678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:53.799551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a1ef9-835d-4693-a9c4-643406e2befa admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.848449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.854926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:53.877920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:53.910756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.911670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.912044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.912375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.913121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.916861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.947157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1310/2620] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:53 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:53.948366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:53.959361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e8ad-2383-4c5a-a9cb-d0da1b948936 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.961963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85de008-9efe-4d1d-94d9-a0f85ba5b409 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:53.967283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.968035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.968314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.968577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] PATH_INFO: `/v3/projects/00814ba4277a4b61a512de1c21d2ec71` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.969195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:53.969767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:53.969919 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00814ba4277a4b61a512de1c21d2ec71)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.970042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:53.970299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:53.970949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:53.972019 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.972739 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:53.985230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1311/2621] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:53.989719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:54.012093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c766919-b1bb-4fe7-9cc1-ac331bbc251c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:54.018614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:54.019325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.019607 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.019882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.020566 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:54.022329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:54.037454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1312/2622] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:00:54.041256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:54.054372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91578148-4dca-42da-9031-4edae1cc00da admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:54.061073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00814ba4277a4b61a512de1c21d2ec71', 'request_id': 'req-013779e8-0c55-46a0-bb4a-51bf6507b2c1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.077726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00814ba4277a4b61a512de1c21d2ec71', 'request_id': 'req-013779e8-0c55-46a0-bb4a-51bf6507b2c1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.104138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00814ba4277a4b61a512de1c21d2ec71', 'request_id': 'req-013779e8-0c55-46a0-bb4a-51bf6507b2c1'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.120698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] The token cache is being invalidate because project 00814ba4277a4b61a512de1c21d2ec71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:54.121327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.122318 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.124743 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1311/2623] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:53 2022] DELETE /identity/v3/projects/00814ba4277a4b61a512de1c21d2ec71 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:54.129455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:54.131299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:54.182210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:54.183375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.183770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.184053 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] PATH_INFO: `/v3/projects/9e08a04a6f3e4f7e8506a64f97bf90e6/users/a280b98a7075418e8fdf29693e2e148e/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.209631 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e08a04a6f3e4f7e8506a64f97bf90e6, user_id=a280b98a7075418e8fdf29693e2e148e, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:54.213418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0f7ea-4040-4506-ace1-7ad095e69323 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:54.215038 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013779e8-0c55-46a0-bb4a-51bf6507b2c1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:54.220353 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:54.221350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.221694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.222129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] PATH_INFO: `/v3/users/28e208b6eb7e49bba714e5ff8f9e6b5e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.223209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e208b6eb7e49bba714e5ff8f9e6b5e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:54.225419 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:54.234709 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1313/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:54 2022] PUT /identity/v3/projects/9e08a04a6f3e4f7e8506a64f97bf90e6/users/a280b98a7075418e8fdf29693e2e148e/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:54.245551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110512-e026-4079-98a8-548dde043419 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.245857 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110512-e026-4079-98a8-548dde043419 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.246083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110512-e026-4079-98a8-548dde043419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.283654 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:54.284765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b8110512-e026-4079-98a8-548dde043419 None None] MFA Rules not processed for user `a280b98a7075418e8fdf29693e2e148e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:54.285854 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b8110512-e026-4079-98a8-548dde043419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:54.292037 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e208b6eb7e49bba714e5ff8f9e6b5e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.294357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e208b6eb7e49bba714e5ff8f9e6b5e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.295448 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e208b6eb7e49bba714e5ff8f9e6b5e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.316535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e208b6eb7e49bba714e5ff8f9e6b5e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.319538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:54.347925 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e208b6eb7e49bba714e5ff8f9e6b5e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:54.367770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1312/2625] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:00:54 2022] DELETE /identity/v3/users/28e208b6eb7e49bba714e5ff8f9e6b5e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:54.398490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1314/2626] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:00:54.411085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:54.412824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:54.508054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70308ac-c820-4a10-a41a-64779505e4af admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:54.515108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:54.516314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.517133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.517585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.675930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:54.678803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe893c4-6872-47a0-946d-7c920f98e984 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:54.694647 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8110512-e026-4079-98a8-548dde043419 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:54.697450 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b8110512-e026-4079-98a8-548dde043419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:54.697724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1313/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:54.852670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8110512-e026-4079-98a8-548dde043419 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:54.970826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:54.971517 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:54.971797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:54.972049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] PATH_INFO: `/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:54.974563 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9ddffaf0ad480ca93ec028a6cc5405)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:54.976520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.057229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9ddffaf0ad480ca93ec028a6cc5405', 'request_id': 'req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:55.079777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9ddffaf0ad480ca93ec028a6cc5405', 'request_id': 'req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:55.092966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9ddffaf0ad480ca93ec028a6cc5405', 'request_id': 'req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:55.118828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:55.135871 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] The token cache is being invalidate because project 7e9ddffaf0ad480ca93ec028a6cc5405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:00:55.136721 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:55.139679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-08f5d4f4-7d2c-4f1e-a438-49a81157aa0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:00:55.148688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1315/2628] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:00:54 2022] DELETE /identity/v3/projects/7e9ddffaf0ad480ca93ec028a6cc5405 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:55.153830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.154597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.154597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.188991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:55.190626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:55.287613 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1314/2629] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:55.296296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.296590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.296888 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.339791 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:55.341954 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:55.378109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1316/2630] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:55.382505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.394986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cabf52-6bb3-4410-890f-8312d5a4d34a None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.405099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.405726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.406011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.406281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.407164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:00:55.407508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.409766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.443181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:55.460998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1315/2631] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 00:00:55.462084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.479583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef73913-05ca-4d2b-9bbc-144acbce596f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.486519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.487719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.488070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.488347 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.489249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.491829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.531361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1317/2632] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:55.536836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.550783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc7f24-f1ed-422c-b4c1-2c0e5e41befc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.627603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.628315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.628653 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.628984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.629701 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.631761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.655286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1316/2633] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:00:55.660445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.681279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e04eb-a86e-409b-8dd8-ce9c1bb553bb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.689090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.689876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.690416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.691157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.692388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.695672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.727593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1318/2634] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:55.736309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.749049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee62f98-36b8-405e-b8ff-095528536119 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.754798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.755579 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.755900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.756176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.756829 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.758672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29373ed1-de78-4a4a-a777-911823989d34 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.791065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1317/2635] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:55.794558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:55.808952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9e2f4-d419-4dbc-9af2-d2dc19600692 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:55.814059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:55.814755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.815020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.815345 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] PATH_INFO: `/v3/projects/69b0d3bcc37d48c385f31d116f702b9c/users/febf2a5863284bcaa94b1904ece9ab15/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.836099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b0d3bcc37d48c385f31d116f702b9c, user_id=febf2a5863284bcaa94b1904ece9ab15, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:55.838835 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:55.854640 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1319/2636] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:55 2022] PUT /identity/v3/projects/69b0d3bcc37d48c385f31d116f702b9c/users/febf2a5863284bcaa94b1904ece9ab15/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:00:55.867452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:55.867777 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:55.868276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:55.905916 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] MFA Rules not processed for user `febf2a5863284bcaa94b1904ece9ab15`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:55.907394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:55.994489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1318/2637] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:55.999707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.078084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1b645-b501-4de1-929d-26f71909a051 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.087026 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.088403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.088885 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.089359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.090535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.094250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.149523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1320/2638] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:56.154659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.169079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2760718-e208-4ed9-8943-f96e5b62597c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.241610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.242246 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.242516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.242776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.243489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.245449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.263521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:56.270741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1319/2639] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:56 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:00:56.275759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.287033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009a626-09b5-4395-bd98-6be194be091b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.292067 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.292695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.292975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.293245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.293903 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.295747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.310108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1321/2640] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:56.314773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.328085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bce-2bef-49d4-8206-6fb3cd29c2c2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.335697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.336576 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.336912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.337442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.338786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.342477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be19534-25ba-4922-8f56-1d6a99523a37 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.359865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1320/2641] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:56.364418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.375868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb3d37-a96f-44f3-a005-921ca6972ef0 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.380996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.381679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.381956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.382226 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] PATH_INFO: `/v3/projects/1015b3d8d45d4350821f7f888e99a410/users/291f68ecf1fa48a58d7f86ee351d48f3/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.398599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1015b3d8d45d4350821f7f888e99a410, user_id=291f68ecf1fa48a58d7f86ee351d48f3, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.401341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.417758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1322/2642] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:56 2022] PUT /identity/v3/projects/1015b3d8d45d4350821f7f888e99a410/users/291f68ecf1fa48a58d7f86ee351d48f3/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:56.430544 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.430819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.430996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.463677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] MFA Rules not processed for user `291f68ecf1fa48a58d7f86ee351d48f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:56.465092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:56.542534 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1321/2643] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:56.553065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.617805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:56.635463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa021bb-b31f-4d0f-99b5-3149b2d14d53 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.641797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.642464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.642728 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.642996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.643666 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.645634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.676902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1323/2644] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:00:56 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:56.681425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.693410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3eb352-1afb-49c9-8026-db7b4523786c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.787940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.787940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.787940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.787940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.788322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.788322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.808410 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1322/2645] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:00:56 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:00:56.812531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.823159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c50cc-54a4-4aff-9068-e22efa6513e8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.827975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.828670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.828949 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.829220 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.829878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.831531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.843690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1324/2646] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:56.848047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.861758 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8dea5-f36e-49c2-8eab-d5a08a575ae0 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:56.866413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:56.867079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:56.867399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:56.867671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] PATH_INFO: `/v3/projects/ac32f65cf66f466e9915cfabee3b624a/users/26c983a5578949b3a9819c4cca4fd7ac/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:56.885066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac32f65cf66f466e9915cfabee3b624a, user_id=26c983a5578949b3a9819c4cca4fd7ac, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:56.887792 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:56.907602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1323/2647] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:56 2022] PUT /identity/v3/projects/ac32f65cf66f466e9915cfabee3b624a/users/26c983a5578949b3a9819c4cca4fd7ac/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:56.914662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:56.927853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e73ca-257c-44aa-82e0-bff9987bc5eb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:57.004320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:57.005771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:57.006243 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:57.006745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:57.007921 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:57.011493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7e7f6-c0e7-4106-9f59-1474bff5e38b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:57.025979 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1325/2648] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:00:57.030495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:57.042222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb1b0-edeb-47b2-992f-67efe9548881 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:57.047795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:57.048575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:57.048913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:57.049245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] PATH_INFO: `/v3/projects/ac32f65cf66f466e9915cfabee3b624a/users/26c983a5578949b3a9819c4cca4fd7ac/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:57.051429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac32f65cf66f466e9915cfabee3b624a, user_id=26c983a5578949b3a9819c4cca4fd7ac, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:57.054441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:57.075524 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1324/2649] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:00:57 2022] PUT /identity/v3/projects/ac32f65cf66f466e9915cfabee3b624a/users/26c983a5578949b3a9819c4cca4fd7ac/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:00:57.087706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:57.087942 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:57.088246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:57.118695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] MFA Rules not processed for user `26c983a5578949b3a9819c4cca4fd7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:00:57.119960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:57.195872 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1326/2650] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:00:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:00:57.208057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:57.210242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:57.247168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:57.301582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4716-8d12-402d-83f6-c039dae168f7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:57.305425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:57.306052 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:57.306316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:57.306578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:57.316578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:57.318100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:57.328922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1325/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:00:57.374425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:00:57.376059 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:00:57.388626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0dfcb-127d-4d94-9bf0-8c7f5b2f0d18 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:00:57.392813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:00:57.393438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:00:57.393714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:00:57.393996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:00:57.441901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:00:57.461995 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:00:57.463359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:00:57.474302 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1327/2652] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:00:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:02.660109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:02.660109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:02.759394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97202f-4405-4e34-8f08-a5bfc745b938 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:02.764042 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:02.764677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:02.764947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:02.765209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:02.767656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:02.776975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:02.902018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:02.903528 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2409-ae82-4967-985f-8540889dc308 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:02.930364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1326/2653] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:02.930685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02314e03-db34-4657-aedf-b6b2f9b6ee23 service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:02.935838 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:02.936507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:02.936789 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:02.937061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] PATH_INFO: `/v3/users/febf2a5863284bcaa94b1904ece9ab15` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:02.938239 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=febf2a5863284bcaa94b1904ece9ab15)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:02.940901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.001795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febf2a5863284bcaa94b1904ece9ab15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.002858 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febf2a5863284bcaa94b1904ece9ab15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.013167 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febf2a5863284bcaa94b1904ece9ab15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.023836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febf2a5863284bcaa94b1904ece9ab15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.041582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febf2a5863284bcaa94b1904ece9ab15'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.048554 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1328/2654] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:02 2022] DELETE /identity/v3/users/febf2a5863284bcaa94b1904ece9ab15 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:03.053795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.054476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.054854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.068647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:03.106283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:03.108093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.190680 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1327/2655] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:03.200783 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:03.202180 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.234073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:03.272648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2516a3-9373-4bbb-acbe-e8e3bb11ce2c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:03.276403 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:03.277408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.277408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.277616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.286641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:03.288203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.291149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:03.292951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9c5e6-e8c9-42c8-b48d-349ce3446a80 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.302619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaf74e-ae6d-4d9a-91b1-469a1451d278 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:03.304268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1329/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:03.316975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:03.317672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.317996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.318287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] PATH_INFO: `/v3/users/a280b98a7075418e8fdf29693e2e148e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.319836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a280b98a7075418e8fdf29693e2e148e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:03.321928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.386363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a280b98a7075418e8fdf29693e2e148e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.387452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a280b98a7075418e8fdf29693e2e148e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.388212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a280b98a7075418e8fdf29693e2e148e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.398050 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a280b98a7075418e8fdf29693e2e148e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.423831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a280b98a7075418e8fdf29693e2e148e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.442814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1328/2657] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:03 2022] DELETE /identity/v3/users/a280b98a7075418e8fdf29693e2e148e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:03.444893 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.445214 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.445500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.469989 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:03.472070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.545616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1330/2658] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:03.562089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:03.563855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.652879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:03.654324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:03.664960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50497e0-7c48-4562-beda-d93a29cbe440 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:03.673092 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:03.674261 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.674722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.675193 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.682710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e283b0-2108-42ad-adcf-6bd4798128a7 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:03.688118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:03.688429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:03.688933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.689188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.689487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] PATH_INFO: `/v3/projects/69b0d3bcc37d48c385f31d116f702b9c` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.689813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.690671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b0d3bcc37d48c385f31d116f702b9c)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:03.692646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.701714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1329/2659] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:03.760770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b0d3bcc37d48c385f31d116f702b9c', 'request_id': 'req-8306823b-4c69-4150-8a8c-11b2d16bb159'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.769585 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b0d3bcc37d48c385f31d116f702b9c', 'request_id': 'req-8306823b-4c69-4150-8a8c-11b2d16bb159'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.776634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b0d3bcc37d48c385f31d116f702b9c', 'request_id': 'req-8306823b-4c69-4150-8a8c-11b2d16bb159'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.793649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] The token cache is being invalidate because project 69b0d3bcc37d48c385f31d116f702b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:03.794338 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.794590 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.796156 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1331/2660] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:03 2022] DELETE /identity/v3/projects/69b0d3bcc37d48c385f31d116f702b9c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:03.801286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:03.813029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cf382-9de0-45e4-b293-8cb66ffef85e service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:03.825360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:03.884910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:03.885561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:03.885836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:03.886105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] PATH_INFO: `/v3/users/291f68ecf1fa48a58d7f86ee351d48f3` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:03.887348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291f68ecf1fa48a58d7f86ee351d48f3)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:03.890081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:03.944274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291f68ecf1fa48a58d7f86ee351d48f3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.945043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291f68ecf1fa48a58d7f86ee351d48f3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.945894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291f68ecf1fa48a58d7f86ee351d48f3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.955134 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291f68ecf1fa48a58d7f86ee351d48f3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.973581 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291f68ecf1fa48a58d7f86ee351d48f3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:03.991289 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1330/2661] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:03 2022] DELETE /identity/v3/users/291f68ecf1fa48a58d7f86ee351d48f3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:04.312107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:04.314125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:04.329339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306823b-4c69-4150-8a8c-11b2d16bb159 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:04.404687 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:04.415257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:04.416146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:04.416461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:04.416754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] PATH_INFO: `/v3/projects/9e08a04a6f3e4f7e8506a64f97bf90e6` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:04.418051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e08a04a6f3e4f7e8506a64f97bf90e6)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:04.421508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:04.452527 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:04.454264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:04.475812 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1130e7-2b80-4791-afc0-286f50f00537 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:04.486292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e08a04a6f3e4f7e8506a64f97bf90e6', 'request_id': 'req-c7923aa4-9b14-40eb-a7b9-3f077d74a274'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.493013 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e08a04a6f3e4f7e8506a64f97bf90e6', 'request_id': 'req-c7923aa4-9b14-40eb-a7b9-3f077d74a274'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.498840 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e08a04a6f3e4f7e8506a64f97bf90e6', 'request_id': 'req-c7923aa4-9b14-40eb-a7b9-3f077d74a274'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.513912 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] The token cache is being invalidate because project 9e08a04a6f3e4f7e8506a64f97bf90e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:04.514246 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.515000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.517384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1332/2662] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:04 2022] DELETE /identity/v3/projects/9e08a04a6f3e4f7e8506a64f97bf90e6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:04.559627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:04.560376 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:04.560678 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:04.560947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] PATH_INFO: `/v3/projects/1015b3d8d45d4350821f7f888e99a410` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:04.562235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1015b3d8d45d4350821f7f888e99a410)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:04.564460 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:04.617371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:04.632429 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1015b3d8d45d4350821f7f888e99a410', 'request_id': 'req-80a18e23-6856-4162-9e29-b3515b56f816'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.642051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1015b3d8d45d4350821f7f888e99a410', 'request_id': 'req-80a18e23-6856-4162-9e29-b3515b56f816'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.653899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1015b3d8d45d4350821f7f888e99a410', 'request_id': 'req-80a18e23-6856-4162-9e29-b3515b56f816'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.669014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] The token cache is being invalidate because project 1015b3d8d45d4350821f7f888e99a410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:04.669733 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.670322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.673435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1331/2663] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:04 2022] DELETE /identity/v3/projects/1015b3d8d45d4350821f7f888e99a410 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:04.679761 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:04.680509 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:04.697588 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7923aa4-9b14-40eb-a7b9-3f077d74a274 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:04.776806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:04.777985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:04.778555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:04.779105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] PATH_INFO: `/v3/users/26c983a5578949b3a9819c4cca4fd7ac` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:04.785075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c983a5578949b3a9819c4cca4fd7ac)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:04.789134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:04.888045 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c983a5578949b3a9819c4cca4fd7ac'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.889811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c983a5578949b3a9819c4cca4fd7ac'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.898244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c983a5578949b3a9819c4cca4fd7ac'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.908508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c983a5578949b3a9819c4cca4fd7ac'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.932642 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-04d76148-3e95-49d5-a941-01846437943c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c983a5578949b3a9819c4cca4fd7ac'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:04.945976 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:04.955381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1333/2664] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:04 2022] DELETE /identity/v3/users/26c983a5578949b3a9819c4cca4fd7ac => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:05.316096 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:05.321108 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:05.340100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a18e23-6856-4162-9e29-b3515b56f816 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:05.431843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:05.432735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.434208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.434208 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] PATH_INFO: `/v3/projects/ac32f65cf66f466e9915cfabee3b624a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.434991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac32f65cf66f466e9915cfabee3b624a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:05.437217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:05.503083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac32f65cf66f466e9915cfabee3b624a', 'request_id': 'req-878269c3-4679-4660-a8fb-a28ea192c5eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:05.510756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac32f65cf66f466e9915cfabee3b624a', 'request_id': 'req-878269c3-4679-4660-a8fb-a28ea192c5eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:05.522291 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac32f65cf66f466e9915cfabee3b624a', 'request_id': 'req-878269c3-4679-4660-a8fb-a28ea192c5eb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:05.535896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] The token cache is being invalidate because project ac32f65cf66f466e9915cfabee3b624a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:05.536199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:05.536842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-878269c3-4679-4660-a8fb-a28ea192c5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:05.538360 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1332/2665] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:05 2022] DELETE /identity/v3/projects/ac32f65cf66f466e9915cfabee3b624a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:05.545755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.546039 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.546923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.603757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:05.604998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:05.676159 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1334/2666] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:05.685849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.686962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.686962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.708868 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:05.710126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:05.720677 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1333/2667] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:05.726528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:05.740943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47d2dc-84f0-42ec-80d9-01791edd835a None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:05.749959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:05.751081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.751544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.752014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.753332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:05.753614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:05.756146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:05.770646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1335/2668] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 00:01:05.775169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:05.788439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04cd6b-c12b-489a-853f-fd718b2ccd00 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:05.794081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:05.795126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.795645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.796106 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.796982 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:05.799272 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:05.846765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1334/2669] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:05 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:05.853318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:05.869328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda111-d134-4447-953f-3e9f39d658d2 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:05.932368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:05.933054 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.933260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.933536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.934188 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:05.936349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:05.967020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1336/2670] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:05 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:05.969457 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:05.984802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bb803-dfa8-4b97-94e9-993df5071174 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:05.990137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:05.990790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:05.991063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:05.991359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:05.992026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:05.993641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.008199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1335/2671] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:06.008822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.029533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945e775-c114-4e08-b339-12bcd4685e34 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.036372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.037112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.037390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.037665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.038324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.039988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7419a-945c-4f64-9b9a-114bdebef86b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.053007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1337/2672] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:06.060647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.078749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9802d-7d62-4bcd-a99b-84327be5922e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.089767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.090748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.091186 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.092022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] PATH_INFO: `/v3/projects/88dfe0de142e4cabaffb3baf6417a0f1/users/9559433738f64151953938c48f560548/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.106880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:06.113820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dfe0de142e4cabaffb3baf6417a0f1, user_id=9559433738f64151953938c48f560548, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.116912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.137125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1336/2673] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:06 2022] PUT /identity/v3/projects/88dfe0de142e4cabaffb3baf6417a0f1/users/9559433738f64151953938c48f560548/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:06.150318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.150615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.150892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.196948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] MFA Rules not processed for user `9559433738f64151953938c48f560548`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:06.198480 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:06.249968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:06.359849 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1338/2674] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:06.365696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.499144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd271-0dad-4867-b552-7bcae26d9530 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.506089 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.506910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.507352 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.508722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.509554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.511798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.563288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1337/2675] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:06 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:06.567856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.582383 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c801d-d1d0-42e9-96da-ec1daeb517bb None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.694421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.695308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.695530 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.695971 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.696463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.698366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.722297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.724203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:06.729907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1339/2676] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:06 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:06.734152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.741444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c40f8-8692-46ee-b739-bcacf9e113fd admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.744440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5be63-e509-4921-8be9-3014365d75b4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.746299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.746935 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.747209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.747542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] PATH_INFO: `/v3/users/0d81b048f7e64c53894ad9e9ee33f518` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.748837 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d81b048f7e64c53894ad9e9ee33f518)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.749633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.750234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.750514 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.750813 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.750926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.751501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.753069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.767917 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1340/2677] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:06.772122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.784361 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b958f-b10c-46f2-babd-2771ab5c2679 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.789745 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.790362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.790638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.790907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.791597 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.793175 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.799967 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d81b048f7e64c53894ad9e9ee33f518'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:06.800625 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d81b048f7e64c53894ad9e9ee33f518'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:06.801221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d81b048f7e64c53894ad9e9ee33f518'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:06.807251 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1341/2678] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:06.810110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:06.810302 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d81b048f7e64c53894ad9e9ee33f518'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:06.811990 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:06.823891 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b618fcd1-8587-495c-96b7-0f237a36bbfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d81b048f7e64c53894ad9e9ee33f518'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:06.827249 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a4e9c-5636-494a-92d7-985c611b772c admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:06.833438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:06.834122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.834402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.834828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] PATH_INFO: `/v3/projects/fae2ea9de119480ab4caa27f7757d03a/users/e55205b4072540f7a2e59c61919cfbe2/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.837439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1338/2679] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:06 2022] DELETE /identity/v3/users/0d81b048f7e64c53894ad9e9ee33f518 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:06.852472 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae2ea9de119480ab4caa27f7757d03a, user_id=e55205b4072540f7a2e59c61919cfbe2, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:06.855064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:06.869559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1342/2680] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:06 2022] PUT /identity/v3/projects/fae2ea9de119480ab4caa27f7757d03a/users/e55205b4072540f7a2e59c61919cfbe2/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:06.881142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:06.881432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:06.881708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:06.913279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] MFA Rules not processed for user `e55205b4072540f7a2e59c61919cfbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:06.915442 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:06.950516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:06.987172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1339/2681] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:06.991972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.036322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.037747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.075358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0ce11-60ae-4463-9eca-dbbe0c48494a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.081453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.082102 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.082375 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.082646 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.083315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.085244 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.111723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:07.120859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8866cc1-cb1d-40c2-a368-0beb83657535 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.123657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1343/2682] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:06 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:07.128811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.202039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.202675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.202940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.203207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] PATH_INFO: `/v3/projects/ffc5ce52ae574b82bda143e766bcec80` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.204638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc5ce52ae574b82bda143e766bcec80)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.206615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.218063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb21cf-87dc-4f25-861a-4b01244f74de admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.223660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.224289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.224591 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.224848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.225494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.227484 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.253805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1344/2683] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:07.255295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.258177 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.266691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc5ce52ae574b82bda143e766bcec80', 'request_id': 'req-eb0aead2-f267-4b1c-911b-bb7f91e94708'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:07.273790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc5ce52ae574b82bda143e766bcec80', 'request_id': 'req-eb0aead2-f267-4b1c-911b-bb7f91e94708'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:07.282061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc5ce52ae574b82bda143e766bcec80', 'request_id': 'req-eb0aead2-f267-4b1c-911b-bb7f91e94708'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:07.294305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] The token cache is being invalidate because project ffc5ce52ae574b82bda143e766bcec80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:07.294610 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:07.295138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:07.296729 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1340/2684] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:07 2022] DELETE /identity/v3/projects/ffc5ce52ae574b82bda143e766bcec80 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:07.297538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.299100 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.315423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:07.376173 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aead2-f267-4b1c-911b-bb7f91e94708 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.381250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.381955 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.382235 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.382493 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.383147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.384878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb743c-3219-4084-8261-598beea7e744 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.387315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33001b0-8dc6-4d32-90bb-b3e564708602 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.401947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1341/2685] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:07.401947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.401947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.401947 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.424383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:07.426632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.436285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1342/2686] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:07.437245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.450999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed189ac-d686-49bb-9db2-a56118b702e3 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.457763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.458550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.458827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.459163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] PATH_INFO: `/v3/projects/9073f5ea965b4ab28efce6031619860d/users/432c2b126b8541afafeefc56b7b9b7e3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.469307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.470198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.470542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.470819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.473257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.480585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9073f5ea965b4ab28efce6031619860d, user_id=432c2b126b8541afafeefc56b7b9b7e3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.483570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb7d20-5b29-4af3-b156-5769143ef3f3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.488304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.490341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.503200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1343/2687] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:07 2022] PUT /identity/v3/projects/9073f5ea965b4ab28efce6031619860d/users/432c2b126b8541afafeefc56b7b9b7e3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:07.504389 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1345/2688] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:07.504630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.505167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.505501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.514813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.527770 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:07.533663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:07.534991 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.544868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b3825-1a9e-4d1e-9d6f-e893316a391e service placement] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.667487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.668304 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.668580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.668878 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.669582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.671629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.683659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1344/2689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:07.685144 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1346/2690] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:07.688933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.693182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.700473 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b0d3-6bdd-464d-bbd7-01118d2922fd None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.707124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698b285-ec17-4438-a858-6d41ac55c001 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.707474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.708035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.708233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.708508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.709178 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:07.709479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.711142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.713619 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.714988 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.715426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.715729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] PATH_INFO: `/v3/projects/9073f5ea965b4ab28efce6031619860d/users/432c2b126b8541afafeefc56b7b9b7e3/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.718206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9073f5ea965b4ab28efce6031619860d, user_id=432c2b126b8541afafeefc56b7b9b7e3, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.722866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd2f07-67b3-40c4-a0c1-9180e21a17ab admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.726324 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1345/2691] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:07.732221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.741125 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1347/2692] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:07 2022] PUT /identity/v3/projects/9073f5ea965b4ab28efce6031619860d/users/432c2b126b8541afafeefc56b7b9b7e3/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:07.751475 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc90a74-af11-49a2-bc92-053deb708544 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:07.752198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.752733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.753118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.789928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] MFA Rules not processed for user `432c2b126b8541afafeefc56b7b9b7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:07.791343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.875933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:07.876691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:07.877084 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:07.877447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:07.881537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:07.881537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:07.891634 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1348/2693] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:07.896658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:07.904209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.906154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:07.916121 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1346/2694] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:07.921543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:07.990385 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c20b00-65dc-4855-9eed-46d5e75caccf None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.013232 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5fc-fd6d-425f-b94b-4d17d54aade6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.019078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.020268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.020268 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.020603 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.023110 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:08.027525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.033113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.073456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1347/2695] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:07 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:08.076230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.077155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.077551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.077923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.080242 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:08.082461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:08.103378 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fbaf-0b1f-4395-954c-dba6434d99f7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.110279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.110915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.111187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.111501 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.112168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.114384 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.134383 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1348/2696] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:08.136210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:08.148491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae251c-893d-491a-9d00-23835974eb07 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.153608 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.154658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.154658 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.154951 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.155831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.158051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.167071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.168584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965502e-17a8-4dea-a4b6-3f004951cc34 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.173334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1349/2697] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:08.178696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:08.181576 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1349/2698] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:08.191093 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd7288-e18b-4c26-8d1a-05c3793006a0 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.200847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.203143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.203143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.203143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] PATH_INFO: `/v3/projects/8f0e75134c574437943e4e551b4f0883/users/8f4781b08fec4b43bd28b7a8396ddb45/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.219308 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0e75134c574437943e4e551b4f0883, user_id=8f4781b08fec4b43bd28b7a8396ddb45, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.222142 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.237029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1350/2699] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:08 2022] PUT /identity/v3/projects/8f0e75134c574437943e4e551b4f0883/users/8f4781b08fec4b43bd28b7a8396ddb45/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:08.248032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.248295 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.248579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.283318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] MFA Rules not processed for user `8f4781b08fec4b43bd28b7a8396ddb45`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:08.284580 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:08.363426 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1350/2700] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:08.376793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:08.379447 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:08.506535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2d20-6d5a-4df9-b23b-322845f2716e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:08.514390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:08.514627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:08.515003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:08.515426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:08.521560 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:08.527181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:08.530094 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce000b-6e6f-49c0-bfec-6845a4ecb947 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:08.545340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1351/2701] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:09.132190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:09.134076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.234381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c808524-eb8d-4691-ba5c-bfc74b478579 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:09.241146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:09.241956 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:09.242234 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:09.242500 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] PATH_INFO: `/v3/users/68fa4bf415794157b08c4bd9ae644339` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:09.243741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fa4bf415794157b08c4bd9ae644339)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:09.245648 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:09.260350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:09.303479 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fa4bf415794157b08c4bd9ae644339'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:09.304319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fa4bf415794157b08c4bd9ae644339'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:09.313136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fa4bf415794157b08c4bd9ae644339'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:09.322506 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fa4bf415794157b08c4bd9ae644339'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:09.339339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fa4bf415794157b08c4bd9ae644339'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:09.346151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1351/2702] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:09 2022] DELETE /identity/v3/users/68fa4bf415794157b08c4bd9ae644339 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:09.350902 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:09.351577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:09.351577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:09.373024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:09.374260 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.452095 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1352/2703] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:09.463170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:09.466444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.551710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0d2-bbe2-4336-9ce2-d086ecaa3793 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:09.555778 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:09.556422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:09.556695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:09.556978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:09.567469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:09.568972 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:09.582323 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1352/2704] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:09.806450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:09.808136 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.871520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb5d3-c81f-498d-915a-0f2a31aa1744 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:09.876307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:09.877361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:09.877765 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:09.878043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:09.880137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.886702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:09.888291 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:09.924550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c4b91-2096-4fd4-bcb3-7c8d5a7e91de service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:09.937981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:09.938738 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:09.939018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:09.939329 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] PATH_INFO: `/v3/projects/09cc6ad304824962b47f1f2767b29579` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:09.940726 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cc6ad304824962b47f1f2767b29579)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:09.942708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:09.962962 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:09.964372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a6666-ea59-4cc5-b66c-837bd7e87f26 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:09.974423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1353/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:09.998052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cc6ad304824962b47f1f2767b29579', 'request_id': 'req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:10.011869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cc6ad304824962b47f1f2767b29579', 'request_id': 'req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:10.019016 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cc6ad304824962b47f1f2767b29579', 'request_id': 'req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:10.041906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] The token cache is being invalidate because project 09cc6ad304824962b47f1f2767b29579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:10.042200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:10.042906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-dc0f1006-acc7-4550-ad5f-d3f04e9e3e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:10.044453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1353/2706] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:09 2022] DELETE /identity/v3/projects/09cc6ad304824962b47f1f2767b29579 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:10.049692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.049999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.050275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.079724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:10.081183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.156645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=100 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:10.178366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1354/2707] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:10.188290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.188837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.189051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.219021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:10.220819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.232193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1354/2708] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:10.237127 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.248007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2af8-3de8-456e-bdfe-bf3e5ca919d6 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.255110 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.255802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.256091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.256350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.257112 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:10.257424 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.259069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.275587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1355/2709] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:10.279974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.294360 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0295b5-8614-4e08-8501-32e2b59a77ba None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.302193 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.305081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.305707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.306639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.308555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.313704 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.321519 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:10.364006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1355/2710] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:10 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:10.368170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.380474 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb587-6c16-4ef0-af10-a0c5d84d91e1 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.460748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.460748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.460748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.460748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.461199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.461695 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.489039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1356/2711] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:10 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:10.490473 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.503380 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d68ce-09ee-4f3f-b7b1-3faf21aa5782 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.508777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.509393 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.509686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.509948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.510937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.512228 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.523723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1356/2712] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:10.527988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.540073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcef2-ad68-42cd-b7e5-97e38afbae44 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.547398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.547398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.547398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.547398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.547904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.549210 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccf17d-39e3-4b74-8ac9-6d2e391bbfc5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.561921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1357/2713] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:10.566218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.577134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500232-fddf-4a6b-bfe7-e2473f492b2e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.583029 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.583719 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.583997 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.584268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] PATH_INFO: `/v3/projects/50ed76a8467247c4b17af450ebb608fb/users/7fd8b341729b4040813b34104e53e818/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.601093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ed76a8467247c4b17af450ebb608fb, user_id=7fd8b341729b4040813b34104e53e818, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.603807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.618151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1357/2714] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:10 2022] PUT /identity/v3/projects/50ed76a8467247c4b17af450ebb608fb/users/7fd8b341729b4040813b34104e53e818/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:10.629637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.629876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.630058 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.660537 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] MFA Rules not processed for user `7fd8b341729b4040813b34104e53e818`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:10.662115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.711036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.712896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.743823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1358/2715] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:10.756722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:10.758224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.784495 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0a45-98cc-4108-8149-ee2b634cb351 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.788492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.789128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.789410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.789680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.791553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:10.811933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:10.828364 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b940b-ee1c-41d7-a412-aae9d7f71c0d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:10.833064 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:10.833694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:10.833985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:10.834552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:10.846117 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.848000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.857881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1359/2716] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:10.867084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:10.868807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13f48a-1832-4e9f-bb61-45363ac3b194 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:10.879839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1358/2717] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:01:16.190206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:16.190206 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:16.279033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101665-2e08-4bd5-9b3d-5635f338d311 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:16.284810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:16.285282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:16.285571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:16.285847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] PATH_INFO: `/v3/users/8f4781b08fec4b43bd28b7a8396ddb45` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:16.287066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4781b08fec4b43bd28b7a8396ddb45)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:16.289277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:16.343159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4781b08fec4b43bd28b7a8396ddb45'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:16.344004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4781b08fec4b43bd28b7a8396ddb45'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:16.344601 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4781b08fec4b43bd28b7a8396ddb45'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:16.351909 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4781b08fec4b43bd28b7a8396ddb45'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:16.366082 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4781b08fec4b43bd28b7a8396ddb45'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:16.387331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1360/2718] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:16 2022] DELETE /identity/v3/users/8f4781b08fec4b43bd28b7a8396ddb45 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:16.392579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:16.393284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:16.393284 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:16.413877 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:16.415107 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:16.484307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1359/2719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:16.495010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:16.497201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:16.566685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01f6a-ea5c-4bdb-99d4-6b34601521e8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:16.572583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:16.577020 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:16.577857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:16.578237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:16.589148 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:16.591426 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:16.607006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1361/2720] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:01:16.925333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:16.927686 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:16.951847 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:16.958868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956195a4-6622-4655-8037-6a3e9eb47716 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:16.970457 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:16.971153 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:16.971502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:16.972046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] PATH_INFO: `/v3/projects/8f0e75134c574437943e4e551b4f0883` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:16.973205 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0e75134c574437943e4e551b4f0883)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:16.975471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:16.996339 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:16.997905 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.014437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dada3-fd10-4dae-8298-fe09c8e5a0b8 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.021507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:17.031890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0e75134c574437943e4e551b4f0883', 'request_id': 'req-73d20f1c-f907-43d3-87e4-83f53beb138b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.039049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0e75134c574437943e4e551b4f0883', 'request_id': 'req-73d20f1c-f907-43d3-87e4-83f53beb138b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.046823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0e75134c574437943e4e551b4f0883', 'request_id': 'req-73d20f1c-f907-43d3-87e4-83f53beb138b'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.060759 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] The token cache is being invalidate because project 8f0e75134c574437943e4e551b4f0883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:17.061052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.061859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-73d20f1c-f907-43d3-87e4-83f53beb138b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.067269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1360/2721] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:16 2022] DELETE /identity/v3/projects/8f0e75134c574437943e4e551b4f0883 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:17.069727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bdfc7-618f-45eb-ab01-d4e1846b5f33 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.070030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bdfc7-618f-45eb-ab01-d4e1846b5f33 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.070312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bdfc7-618f-45eb-ab01-d4e1846b5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.096260 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-213bdfc7-618f-45eb-ab01-d4e1846b5f33 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:17.097474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-213bdfc7-618f-45eb-ab01-d4e1846b5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.100519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.101163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.101465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.101703 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] PATH_INFO: `/v3/users/9559433738f64151953938c48f560548` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.102848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9559433738f64151953938c48f560548)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.104819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.106440 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1361/2722] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:17.114508 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.114771 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.115009 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.135700 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:17.136796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.146010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1362/2723] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:17.150353 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.155556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9559433738f64151953938c48f560548'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.156941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9559433738f64151953938c48f560548'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.157577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9559433738f64151953938c48f560548'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.159578 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efc061-be52-4b74-a165-9b3f7c557f72 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.164166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.164763 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.165036 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.165164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9559433738f64151953938c48f560548'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.165333 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.165969 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:17.166268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.167876 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.177698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-96bc93e4-10e0-4fae-bd60-2a124717fa61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9559433738f64151953938c48f560548'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:17.179960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1363/2724] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 00:01:17.184866 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.186802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.192340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1362/2725] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:16 2022] DELETE /identity/v3/users/9559433738f64151953938c48f560548 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:17.196562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.196849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.197147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.200105 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6cb72-cc6f-4591-a313-eca4c45e4237 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.224754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:17.226065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.269567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.270196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.270476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.270750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.271445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.273357 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.300566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1363/2726] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:17.306082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1364/2727] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/projects => generated 407 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:17.310172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.310679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.312395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.320957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f587b-db3f-4156-abf1-497ad129e0b2 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.362889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:17.410107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.411434 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.411730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.412285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.412994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.415162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.454257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1364/2728] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/users => generated 304 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:17.459998 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.475090 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c9451-16c1-44a8-9db8-df86e47c50aa admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.483751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.485023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.485533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.486091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.487517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.490702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.506659 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1365/2729] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:17.510667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:17.511277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.526245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48cffb-0645-4de2-9580-8c2c6fcc2ea3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.526610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c17d9e-6523-4c3e-97b7-b33925c2a916 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.530780 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.531483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.531752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.531886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.532020 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.532388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.532687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.532963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.533635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.535343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.544025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.544964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25f40c-b897-4496-94c4-6cbf2045e354 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.552802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1366/2730] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:17.553953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.560727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1365/2731] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:01:17.565039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081f5d-8a7b-4d15-a5cd-8b8002cd29c4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.571490 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.572182 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.572865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.573719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] PATH_INFO: `/v3/projects/e3d331b2f94d43148076b7aab0cf2f09/users/2d55c1b8bec64474b0a26424f8a531fe/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.594482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d331b2f94d43148076b7aab0cf2f09, user_id=2d55c1b8bec64474b0a26424f8a531fe, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.597380 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.614066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1367/2732] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:17 2022] PUT /identity/v3/projects/e3d331b2f94d43148076b7aab0cf2f09/users/2d55c1b8bec64474b0a26424f8a531fe/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:17.629993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.630483 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.630793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.672231 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] MFA Rules not processed for user `2d55c1b8bec64474b0a26424f8a531fe`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:17.674028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.765620 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1366/2733] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:17.771146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.842248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfc889-cca5-4ab0-a3d8-7ce0ef3f1dfc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.848377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.849588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.850012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.850359 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.851109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.853958 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:17.888159 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1368/2734] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/projects => generated 407 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:17.894896 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.905983 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b073291-93ff-44ca-9001-9acbb029c4bd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:17.913814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:17.917109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:17.983455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:17.984118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:17.984418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:17.984688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:17.985328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:17.987421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.007836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:18.011381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1367/2735] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:17 2022] POST /identity/v3/users => generated 310 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:18.012223 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe893fc-2a9f-46c2-975c-59797c258727 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.016691 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.020634 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.021370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.021694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.022026 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] PATH_INFO: `/v3/projects/88dfe0de142e4cabaffb3baf6417a0f1` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.023519 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dfe0de142e4cabaffb3baf6417a0f1)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.027074 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.032152 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d95b1-8e37-46f0-a1ad-14118620508d admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.038608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.039595 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.039947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.040496 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.041684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.043328 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.055501 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1368/2736] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:18.060822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.074071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1f14e-2034-45c4-bf84-5d7471ef1ebd admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.089667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dfe0de142e4cabaffb3baf6417a0f1', 'request_id': 'req-17ecf686-33c3-4360-953b-5c8ee3015a8e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.097234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dfe0de142e4cabaffb3baf6417a0f1', 'request_id': 'req-17ecf686-33c3-4360-953b-5c8ee3015a8e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.106075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dfe0de142e4cabaffb3baf6417a0f1', 'request_id': 'req-17ecf686-33c3-4360-953b-5c8ee3015a8e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.121872 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] The token cache is being invalidate because project 88dfe0de142e4cabaffb3baf6417a0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:18.122168 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.122867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.124517 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1369/2737] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:17 2022] DELETE /identity/v3/projects/88dfe0de142e4cabaffb3baf6417a0f1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:18.129057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.130609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.144638 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:18.154339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.155051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.155541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.155939 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] PATH_INFO: `/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad/users/da4af995a8044b819a1f08a5c238b59e/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.172285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9daf9e0163b04bb4b6c4621274bc09ad, user_id=da4af995a8044b819a1f08a5c238b59e, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.175062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.191487 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1369/2738] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:18 2022] PUT /identity/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad/users/da4af995a8044b819a1f08a5c238b59e/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:18.195737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.197598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.204269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf686-33c3-4360-953b-5c8ee3015a8e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.266357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.267069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.267584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.267854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] PATH_INFO: `/v3/users/e55205b4072540f7a2e59c61919cfbe2` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.269465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e55205b4072540f7a2e59c61919cfbe2)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.271477 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.272368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0eed7-52ce-4579-a955-15e449eba664 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.280349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.281008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.281285 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.281553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.282211 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.283907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.300213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1370/2739] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:18.301349 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.312742 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5371800-b8c6-4c8c-a61c-0d4066689f06 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.318926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.319720 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.320065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.320394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] PATH_INFO: `/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad/users/da4af995a8044b819a1f08a5c238b59e/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.322643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9daf9e0163b04bb4b6c4621274bc09ad, user_id=da4af995a8044b819a1f08a5c238b59e, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.323335 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:18.325706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304d36-8a10-41da-ab56-b07f6e63f042 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.336767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e55205b4072540f7a2e59c61919cfbe2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.337971 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e55205b4072540f7a2e59c61919cfbe2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.339073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e55205b4072540f7a2e59c61919cfbe2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.340758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1371/2740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:18 2022] PUT /identity/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad/users/da4af995a8044b819a1f08a5c238b59e/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:18.353914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.354198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.354475 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.355568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e55205b4072540f7a2e59c61919cfbe2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.368870 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e55205b4072540f7a2e59c61919cfbe2'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.385466 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] MFA Rules not processed for user `da4af995a8044b819a1f08a5c238b59e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:18.386705 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.387511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1370/2741] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:18 2022] DELETE /identity/v3/users/e55205b4072540f7a2e59c61919cfbe2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:18.470344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1372/2742] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:18 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:18.482628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.484097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.556545 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710d823-04e1-4f20-9240-d4b37504f2ed admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.561039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.561861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.562135 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.562404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.587128 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.589253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.604782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1371/2743] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:18.738268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.740390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.828607 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:18.838007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae966d-120e-4134-b9cd-6796f1a73d15 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.841172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:18.848937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.849917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.850279 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.850631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.853068 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:18.869061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac7e3b-0f75-45c9-9506-3dad503ac1c6 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:18.874658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:18.875471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:18.875610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:18.875890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] PATH_INFO: `/v3/projects/fae2ea9de119480ab4caa27f7757d03a` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:18.879275 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae2ea9de119480ab4caa27f7757d03a)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.880109 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.937865 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:18.939508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:18.949649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1372/2744] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:18 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:01:18.957423 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae2ea9de119480ab4caa27f7757d03a', 'request_id': 'req-7b88be02-5cf4-466f-80fe-62f7955295df'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.964879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae2ea9de119480ab4caa27f7757d03a', 'request_id': 'req-7b88be02-5cf4-466f-80fe-62f7955295df'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.971601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae2ea9de119480ab4caa27f7757d03a', 'request_id': 'req-7b88be02-5cf4-466f-80fe-62f7955295df'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.985494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] The token cache is being invalidate because project fae2ea9de119480ab4caa27f7757d03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:18.985860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.986518 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:18.988173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1373/2745] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:18 2022] DELETE /identity/v3/projects/fae2ea9de119480ab4caa27f7757d03a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:18.993417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:19.009098 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6242c3-ad6a-4517-afec-a26b1c5ae159 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:19.076550 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:19.090334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:19.090932 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.091204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.091502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] PATH_INFO: `/v3/users/432c2b126b8541afafeefc56b7b9b7e3` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.092726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=432c2b126b8541afafeefc56b7b9b7e3)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:19.094823 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:19.152218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432c2b126b8541afafeefc56b7b9b7e3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.152970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432c2b126b8541afafeefc56b7b9b7e3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.153709 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432c2b126b8541afafeefc56b7b9b7e3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.161664 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432c2b126b8541afafeefc56b7b9b7e3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.178533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5a3ec730-7d8d-4afe-9197-374818fa9cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432c2b126b8541afafeefc56b7b9b7e3'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.198286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1373/2746] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:18 2022] DELETE /identity/v3/users/432c2b126b8541afafeefc56b7b9b7e3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:19.555112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:19.557015 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:19.573766 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88be02-5cf4-466f-80fe-62f7955295df admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:19.664061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:19.665163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.665444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.665711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] PATH_INFO: `/v3/projects/9073f5ea965b4ab28efce6031619860d` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.667061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9073f5ea965b4ab28efce6031619860d)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:19.669151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:19.729458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9073f5ea965b4ab28efce6031619860d', 'request_id': 'req-3bcd44ba-fe67-40a1-baa3-307429ad45b8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.736438 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9073f5ea965b4ab28efce6031619860d', 'request_id': 'req-3bcd44ba-fe67-40a1-baa3-307429ad45b8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.743445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9073f5ea965b4ab28efce6031619860d', 'request_id': 'req-3bcd44ba-fe67-40a1-baa3-307429ad45b8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.759963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] The token cache is being invalidate because project 9073f5ea965b4ab28efce6031619860d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:19.760257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.760922 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3bcd44ba-fe67-40a1-baa3-307429ad45b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:19.762491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1374/2747] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:19 2022] DELETE /identity/v3/projects/9073f5ea965b4ab28efce6031619860d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:19.768552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.769221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.769221 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.792876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:19.794137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:19.863071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1374/2748] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:19.872999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.873303 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.873581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.888636 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:19.895707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:19.897856 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:19.906527 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1375/2749] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:19.911325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:19.922635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f897-69de-4009-b8b8-75c8dc18891d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:19.928450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:19.929070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.929345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.929617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.930274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:19.930577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:19.932269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:19.951556 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1375/2750] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:19.952813 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:19.963090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b783fd-9928-4284-86be-eb7240bf7f17 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:19.969195 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:19.969829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:19.970279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:19.970546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:19.971190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:19.973142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.007608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1376/2751] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:19 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:20.012538 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:20.035655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3144-2e7b-4218-8658-331469745aa7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:20.117689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:20.118489 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.118763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.119034 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.119712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:20.122086 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.133105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:20.155494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1376/2752] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:20 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:20.156975 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:20.171458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596154d-a5e5-46c1-b1b3-a507fd3103c5 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:20.177657 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:20.178540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.178915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.179210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.180014 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:20.181793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.192853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1377/2753] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:20.197204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:20.210927 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed3335-7f11-476a-88a9-06c0df8fe910 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:20.216984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:20.217806 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.218067 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.218531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.219250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:20.221046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504b438-ff67-421d-9bf5-e639609f607b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.233555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1377/2754] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:20.238255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:20.249408 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7f58e-17d3-4304-bc22-63b787520260 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:20.254564 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:20.255292 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.255624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.255865 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] PATH_INFO: `/v3/projects/eed199c167c8478f845f858b9cb736b3/users/0ea811e3d79c4f8aa7beb28aa0230dc5/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.277468 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed199c167c8478f845f858b9cb736b3, user_id=0ea811e3d79c4f8aa7beb28aa0230dc5, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:20.279733 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.297162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1378/2755] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:20 2022] PUT /identity/v3/projects/eed199c167c8478f845f858b9cb736b3/users/0ea811e3d79c4f8aa7beb28aa0230dc5/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:20.307542 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.307814 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.308248 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.343616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] MFA Rules not processed for user `0ea811e3d79c4f8aa7beb28aa0230dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:20.345004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:20.429165 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1378/2756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:20.443673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:20.446208 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:20.522012 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee490-17d8-4f52-869a-a5f1916bad57 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:20.526218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:20.526852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:20.527127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:20.527431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:20.538003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:20.539610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6290b6a-280e-4371-b320-61dd2b8a115b service glance] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:20.549736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1379/2757] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:29.915129 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:29.916802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:30.005549 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964659c-f03a-4c97-99e6-9811c784fab1 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:30.011390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:30.012133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:30.012539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:30.012838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] PATH_INFO: `/v3/users/2d55c1b8bec64474b0a26424f8a531fe` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:30.014176 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d55c1b8bec64474b0a26424f8a531fe)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:30.016764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:30.084567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d55c1b8bec64474b0a26424f8a531fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.085363 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d55c1b8bec64474b0a26424f8a531fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.086016 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d55c1b8bec64474b0a26424f8a531fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.096646 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d55c1b8bec64474b0a26424f8a531fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.112439 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:30.114412 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d55c1b8bec64474b0a26424f8a531fe'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.133486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1379/2758] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:29 2022] DELETE /identity/v3/users/2d55c1b8bec64474b0a26424f8a531fe => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:30.141755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:30.142463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:30.143186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:30.164846 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:30.166090 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:30.240334 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1380/2759] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:30.251719 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:30.253371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:30.371309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5954-976f-4e96-8a0a-9a153c6a72a5 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:30.377952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:30.378989 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:30.379513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:30.379975 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:30.397461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:30.400171 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:30.417929 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1380/2760] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:30.769688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:30.771372 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:30.786490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34b69-d459-4029-abb0-5c6de666e10c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:30.793999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:30.794670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:30.794937 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:30.795201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] PATH_INFO: `/v3/projects/e3d331b2f94d43148076b7aab0cf2f09` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:30.796474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d331b2f94d43148076b7aab0cf2f09)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:30.798533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:30.850065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:30.858823 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d331b2f94d43148076b7aab0cf2f09', 'request_id': 'req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.869318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d331b2f94d43148076b7aab0cf2f09', 'request_id': 'req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.876567 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d331b2f94d43148076b7aab0cf2f09', 'request_id': 'req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.892186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] The token cache is being invalidate because project e3d331b2f94d43148076b7aab0cf2f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:30.892458 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.893166 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:30.894730 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1381/2761] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:30 2022] DELETE /identity/v3/projects/e3d331b2f94d43148076b7aab0cf2f09 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:30.900767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:30.911138 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b121e2-bb52-48a0-8e9b-a3b89b063bd4 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:30.991798 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:30.992896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:30.992896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:30.993217 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] PATH_INFO: `/v3/users/da4af995a8044b819a1f08a5c238b59e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:30.994131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4af995a8044b819a1f08a5c238b59e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:30.996216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:31.057588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4af995a8044b819a1f08a5c238b59e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.058522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4af995a8044b819a1f08a5c238b59e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.059385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4af995a8044b819a1f08a5c238b59e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.071901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4af995a8044b819a1f08a5c238b59e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.096511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-a2815034-b092-401c-b315-1632355f3753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4af995a8044b819a1f08a5c238b59e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.099877 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:31.116917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1381/2762] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:30 2022] DELETE /identity/v3/users/da4af995a8044b819a1f08a5c238b59e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:31.796948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:31.798676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:31.815237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366d8f-25b9-4fcf-8c8b-979f24d0b621 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:31.883161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:31.883848 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:31.884129 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:31.884397 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] PATH_INFO: `/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:31.885618 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9daf9e0163b04bb4b6c4621274bc09ad)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:31.887593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:31.942141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9daf9e0163b04bb4b6c4621274bc09ad', 'request_id': 'req-b4bb9b6e-682b-4873-b265-baeddbaa5036'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.948929 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9daf9e0163b04bb4b6c4621274bc09ad', 'request_id': 'req-b4bb9b6e-682b-4873-b265-baeddbaa5036'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.957040 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9daf9e0163b04bb4b6c4621274bc09ad', 'request_id': 'req-b4bb9b6e-682b-4873-b265-baeddbaa5036'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.976815 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] The token cache is being invalidate because project 9daf9e0163b04bb4b6c4621274bc09ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:31.977128 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.977818 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-b4bb9b6e-682b-4873-b265-baeddbaa5036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:31.979477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1382/2763] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:31 2022] DELETE /identity/v3/projects/9daf9e0163b04bb4b6c4621274bc09ad => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:31.984710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:31.984996 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:31.985304 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.006666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:32.008004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:32.101485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1382/2764] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:32.111187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.111534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.111805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.134416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:32.135692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:32.145755 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1383/2765] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:32.152627 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.166488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a2c3b-724d-456f-80ec-8e0fdbd834b5 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.172570 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.173245 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.173521 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.173788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.174446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:32.174747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.176484 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.192430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1383/2766] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:32.194953 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.205926 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91db554-4dca-4101-aa2b-669177396e77 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.211725 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.213021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.213021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.213021 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.213464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.215001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.256084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1384/2767] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:32.257796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.276290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae0a1-065d-4643-a76f-cd85b959badc admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.356321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.356948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.357280 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.357533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.358228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.360397 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.383441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1384/2768] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:32.388395 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.405868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d5251-055a-499c-b584-8f954a0814ae admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.414731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.415793 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.416254 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.416713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.418431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.421535 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.444035 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1385/2769] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:32.450898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.470143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae5299-c48a-4361-89cf-800e88d43566 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.479158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.479933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.480565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.480977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.482107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.484672 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fad3b-ff73-45c6-80d7-436a67db3189 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.502817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1385/2770] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:32.504062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.516796 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e2fe-48b3-4ffd-bca9-ee42fd8b85d4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.523096 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.523925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.524293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.524609 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] PATH_INFO: `/v3/projects/d3a7c21f7f0c45cab808263cd9a9b2ed/users/cbc795036702408dbe130fdba2cd8764/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.548269 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a7c21f7f0c45cab808263cd9a9b2ed, user_id=cbc795036702408dbe130fdba2cd8764, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.551414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.555533 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:32.569707 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1386/2771] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:32 2022] PUT /identity/v3/projects/d3a7c21f7f0c45cab808263cd9a9b2ed/users/cbc795036702408dbe130fdba2cd8764/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:32.582977 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.584145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.588241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.627423 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] MFA Rules not processed for user `cbc795036702408dbe130fdba2cd8764`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:32.629498 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:32.690105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:32.726060 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1386/2772] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:32.730608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.799873 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c6ec2-2967-49b4-aac4-b81902a008d7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.804863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.805497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.805772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.806038 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.806688 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.808680 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:32.865305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1387/2773] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:01:32.867950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:32.886948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27065-909f-44b9-9d93-ed299f262395 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:32.993179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:32.994051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:32.994800 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:32.995267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:32.995992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:32.998103 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.027938 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1387/2774] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:32 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:33.034583 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:33.052867 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002cae5-8f08-4e47-bbce-2ab7d85ec41b admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:33.063157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:33.064046 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.064359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.064616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.065335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:33.067192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.080767 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1388/2775] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:33.085702 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:33.097505 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e265-c3dc-4d2b-8348-c9684f13e58d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:33.102782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:33.103510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.103790 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.104061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] PATH_INFO: `/v3/projects/babb10a2972849c3b0dc8692bbbc8a46/users/ef6dc488261c4e999640ec344f9cab80/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.121617 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=babb10a2972849c3b0dc8692bbbc8a46, user_id=ef6dc488261c4e999640ec344f9cab80, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:33.124631 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.145154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1388/2776] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:33 2022] PUT /identity/v3/projects/babb10a2972849c3b0dc8692bbbc8a46/users/ef6dc488261c4e999640ec344f9cab80/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:33.149497 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:33.160906 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1638c0-730d-4dca-87cb-c257bd4d533f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:33.236841 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:33.237436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.237811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.238169 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.238882 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:33.240651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a02d0-2897-4299-89f4-1738542aec3e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.252963 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1389/2777] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:33.257855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:33.274163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13950d-d862-4007-b46a-8f1189092fc8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:33.280952 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:33.282567 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.283204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.283887 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] PATH_INFO: `/v3/projects/babb10a2972849c3b0dc8692bbbc8a46/users/ef6dc488261c4e999640ec344f9cab80/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.289228 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] RBAC: Authorizing `identity:create_grant(project_id=babb10a2972849c3b0dc8692bbbc8a46, user_id=ef6dc488261c4e999640ec344f9cab80, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:33.295198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.316994 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1389/2778] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:33 2022] PUT /identity/v3/projects/babb10a2972849c3b0dc8692bbbc8a46/users/ef6dc488261c4e999640ec344f9cab80/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:33.328629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.329154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.329382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.364399 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] MFA Rules not processed for user `ef6dc488261c4e999640ec344f9cab80`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:33.366834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:33.433322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:33.457829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1390/2779] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:33.472499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:33.474584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:33.604287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b92-5676-4c02-b86d-918511d1236e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:33.610861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:33.611859 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:33.612132 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:33.612438 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:33.628830 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:33.632311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180f868-c6bd-4a48-b804-ccc098f0333a service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:33.643012 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1390/2780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:34.363861 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:34.440756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522c32b-673a-448c-8ced-6e9483c6d252 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:34.445864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:34.446528 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:34.446806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:34.447073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] PATH_INFO: `/v3/users/cbc795036702408dbe130fdba2cd8764` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:34.448343 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc795036702408dbe130fdba2cd8764)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:34.450286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:34.507740 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc795036702408dbe130fdba2cd8764'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:34.508714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc795036702408dbe130fdba2cd8764'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:34.518424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc795036702408dbe130fdba2cd8764'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:34.528411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc795036702408dbe130fdba2cd8764'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:34.542279 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc795036702408dbe130fdba2cd8764'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:34.549986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1391/2781] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:34 2022] DELETE /identity/v3/users/cbc795036702408dbe130fdba2cd8764 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:01:34.554866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:34.555476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:34.555628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:34.577347 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:34.578726 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:34.601351 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:34.671024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1391/2782] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:34.681751 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:34.683445 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:34.759911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18082f1d-f389-4a00-b8f2-58b5730d7b81 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:34.764306 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:34.765424 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:34.765673 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:34.765944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:34.778692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:34.780187 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:34.791069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1392/2783] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:35.131571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:35.133059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:35.152046 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367935b2-0e8c-48b1-89de-90932bc1fb9c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:35.154430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:35.155522 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:35.155912 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:35.156189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] PATH_INFO: `/v3/projects/d3a7c21f7f0c45cab808263cd9a9b2ed` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:35.157694 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a7c21f7f0c45cab808263cd9a9b2ed)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:35.159943 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:35.224277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a7c21f7f0c45cab808263cd9a9b2ed', 'request_id': 'req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.231495 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a7c21f7f0c45cab808263cd9a9b2ed', 'request_id': 'req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.241933 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a7c21f7f0c45cab808263cd9a9b2ed', 'request_id': 'req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.254739 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] The token cache is being invalidate because project d3a7c21f7f0c45cab808263cd9a9b2ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:35.255033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.255816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.257393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1392/2784] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:35 2022] DELETE /identity/v3/projects/d3a7c21f7f0c45cab808263cd9a9b2ed => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:35.261626 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:35.272894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b9e3-b187-465c-9e8f-7ce01b2a4e30 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:35.282753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:35.345433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:35.346154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:35.346432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:35.346851 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] PATH_INFO: `/v3/users/ef6dc488261c4e999640ec344f9cab80` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:35.348252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6dc488261c4e999640ec344f9cab80)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:35.350319 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:35.412546 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6dc488261c4e999640ec344f9cab80'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.413293 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6dc488261c4e999640ec344f9cab80'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.425165 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6dc488261c4e999640ec344f9cab80'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.435661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6dc488261c4e999640ec344f9cab80'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.450910 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-38c0b8c1-cf17-4cdb-9dd5-964c827e3d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6dc488261c4e999640ec344f9cab80'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.458540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1393/2785] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:35 2022] DELETE /identity/v3/users/ef6dc488261c4e999640ec344f9cab80 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:35.770399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:35.772124 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:35.787307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8d2a-c34a-48cd-8c8e-7c984cfcba48 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:35.868690 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:35.869468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:35.870078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:35.870590 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] PATH_INFO: `/v3/projects/babb10a2972849c3b0dc8692bbbc8a46` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:35.872486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=babb10a2972849c3b0dc8692bbbc8a46)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:35.875767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:35.939137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:35.941510 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'babb10a2972849c3b0dc8692bbbc8a46', 'request_id': 'req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.951820 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'babb10a2972849c3b0dc8692bbbc8a46', 'request_id': 'req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.964313 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'babb10a2972849c3b0dc8692bbbc8a46', 'request_id': 'req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.976212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] The token cache is being invalidate because project babb10a2972849c3b0dc8692bbbc8a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:35.976500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.977163 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-0f4f3e12-63f5-4bcf-8524-105a1e7aa781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:35.978716 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1393/2786] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:35 2022] DELETE /identity/v3/projects/babb10a2972849c3b0dc8692bbbc8a46 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:35.983859 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:35.984146 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:35.984416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.006064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:36.007344 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:36.091237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1394/2787] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:36.099539 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.099817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.100085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.125057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:36.126348 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:36.134685 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1394/2788] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:36.139006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.149718 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94065de-b6a3-427c-9c46-5c6aa85afa6f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.154255 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.154863 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.155133 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.155453 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.156136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:36.156428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.158072 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.175082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1395/2789] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:36.180540 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.196013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27330121-4b5d-42fa-8489-edd41f649f6f None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.205218 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.206078 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.206450 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.206811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.207588 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.209512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.246088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1395/2790] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:36.252532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.267649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0310-5482-485e-8df4-16aa9976ab92 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.289665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:36.334044 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.334670 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.334945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.335237 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.335881 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.337808 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.362271 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1396/2791] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:36.366592 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.384391 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6e664-d1a9-4f60-b915-633085639707 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.389408 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.390373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.390649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.390924 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.391697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.393327 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.404682 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1396/2792] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:01:36.409586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.421800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb5e5-f458-4f21-b5ea-f98e2e8a6539 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.427531 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.428145 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.428415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.428679 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.429327 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.430957 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518013a-bfc1-4233-ab4c-9cf032258dae admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.443477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1397/2793] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:36.450822 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.470276 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df9ab3-6120-4408-a320-baae0566376e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.475628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.476293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.476571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.476836 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] PATH_INFO: `/v3/projects/d976b3938eb24f6f8cd3133aa1fdce3e/users/0e77abfc39a242b7afc8b46535ab5766/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.493783 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d976b3938eb24f6f8cd3133aa1fdce3e, user_id=0e77abfc39a242b7afc8b46535ab5766, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.496508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.511322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1397/2794] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:36 2022] PUT /identity/v3/projects/d976b3938eb24f6f8cd3133aa1fdce3e/users/0e77abfc39a242b7afc8b46535ab5766/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:36.524034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.524321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.524569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.559520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] MFA Rules not processed for user `0e77abfc39a242b7afc8b46535ab5766`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:36.560883 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:36.628194 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1398/2795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:36.633053 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.704081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfa7c3-f4fc-426b-a9e0-9153ca108c36 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.709219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.709827 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.710158 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.710428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.711071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.713003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.742146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1398/2796] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:36.746470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.757536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e51f01-37ba-45d4-863f-a2db6688529f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.820161 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.820779 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.821081 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.821322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.821944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.823904 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.845831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1399/2797] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:36 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:36.849922 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.862192 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8601500-bd6a-41f0-a0d5-24eb35dd6f72 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.867623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.868237 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.868511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.868776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.869420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.871085 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.886624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1399/2798] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:36.889874 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.900911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaab39-c26e-4242-8f95-53c6262cf4a9 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:36.905498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:36.906154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:36.906422 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:36.906690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] PATH_INFO: `/v3/projects/41dc21dba93942768680f772d2b366c9/users/5c3d9a5a2c9b4b36af5ad99dcab09d30/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:36.922960 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dc21dba93942768680f772d2b366c9, user_id=5c3d9a5a2c9b4b36af5ad99dcab09d30, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:36.925677 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:36.928298 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:36.944320 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1400/2799] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:36 2022] PUT /identity/v3/projects/41dc21dba93942768680f772d2b366c9/users/5c3d9a5a2c9b4b36af5ad99dcab09d30/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:36.950018 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:36.961984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a76f2b-ea72-46ca-9f81-f78c321bd242 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:37.013167 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:37.031779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:37.032406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:37.032710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:37.032981 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:37.033660 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:37.035462 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2aa4a-c36e-4226-a1c6-d100cd469cd9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:37.048562 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1400/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:37.052966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:37.065735 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92cd66-4d88-401f-a770-618e05f03f6a admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:37.070806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:37.071538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:37.071817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:37.072499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] PATH_INFO: `/v3/projects/41dc21dba93942768680f772d2b366c9/users/5c3d9a5a2c9b4b36af5ad99dcab09d30/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:37.074070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dc21dba93942768680f772d2b366c9, user_id=5c3d9a5a2c9b4b36af5ad99dcab09d30, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:37.077652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:37.090299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1401/2801] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:37 2022] PUT /identity/v3/projects/41dc21dba93942768680f772d2b366c9/users/5c3d9a5a2c9b4b36af5ad99dcab09d30/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:37.102011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:37.102288 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:37.102561 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:37.139779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] MFA Rules not processed for user `5c3d9a5a2c9b4b36af5ad99dcab09d30`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:37.144160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:37.232286 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1401/2802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:37.247350 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:37.249388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:37.324830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20616e-51cf-4424-856a-cb5458dc51e1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:37.329023 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:37.329628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:37.329923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:37.330179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:37.395263 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:37.396699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:37.406097 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1402/2803] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:41.808533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:41.819507 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaebe62-2e82-4612-b4c6-bc212d404357 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:41.823456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:41.824325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:41.824621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:41.824888 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:41.835073 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:41.836555 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88d60d-955f-49c1-b40b-817273d92188 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:41.848399 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1402/2804] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:01:47.401608 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:INSERT=2,keystone:SELECT=55 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:49.027661 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:49.110784 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfd9ca-a832-4f2e-ae7b-7d532502425f service cinder] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:49.117932 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:49.118628 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:49.118998 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:49.119309 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] PATH_INFO: `/v3/users/0e77abfc39a242b7afc8b46535ab5766` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:49.120503 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e77abfc39a242b7afc8b46535ab5766)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:49.122470 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:49.182843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e77abfc39a242b7afc8b46535ab5766'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.183507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e77abfc39a242b7afc8b46535ab5766'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.190606 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e77abfc39a242b7afc8b46535ab5766'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.199933 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e77abfc39a242b7afc8b46535ab5766'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.214022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e77abfc39a242b7afc8b46535ab5766'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.220359 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1403/2805] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:49 2022] DELETE /identity/v3/users/0e77abfc39a242b7afc8b46535ab5766 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:49.225910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:49.226453 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:49.226578 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:49.249504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:49.250754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:49.325553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1403/2806] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:49.336374 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:49.338093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:49.402713 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637d495-86e9-4b29-b524-cfddcca7b0d1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:49.407149 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:49.407805 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:49.408086 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:49.408348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:49.418901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:49.420461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:49.431069 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1404/2807] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:49.737949 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:49.740224 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:49.762803 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5ebd5-26bc-46c9-a7a1-3c694f41e3ea None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:49.771615 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:49.772413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:49.772817 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:49.773323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] PATH_INFO: `/v3/projects/d976b3938eb24f6f8cd3133aa1fdce3e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:49.775500 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d976b3938eb24f6f8cd3133aa1fdce3e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:49.778283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:49.845009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd976b3938eb24f6f8cd3133aa1fdce3e', 'request_id': 'req-99136676-d67e-461e-8b82-b09d12c486b8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.846047 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:49.853609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd976b3938eb24f6f8cd3133aa1fdce3e', 'request_id': 'req-99136676-d67e-461e-8b82-b09d12c486b8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.865584 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd976b3938eb24f6f8cd3133aa1fdce3e', 'request_id': 'req-99136676-d67e-461e-8b82-b09d12c486b8'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.877015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] The token cache is being invalidate because project d976b3938eb24f6f8cd3133aa1fdce3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:49.877297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.877986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:49.879915 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1404/2808] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:49 2022] DELETE /identity/v3/projects/d976b3938eb24f6f8cd3133aa1fdce3e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:49.883788 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:49.898131 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9aed3-72f5-4116-a3af-ab738c779e65 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:49.931094 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:49.983820 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:49.984802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:49.985123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:49.985540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] PATH_INFO: `/v3/users/5c3d9a5a2c9b4b36af5ad99dcab09d30` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:49.987387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3d9a5a2c9b4b36af5ad99dcab09d30)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:49.989993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.053121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3d9a5a2c9b4b36af5ad99dcab09d30'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.053801 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3d9a5a2c9b4b36af5ad99dcab09d30'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.062335 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3d9a5a2c9b4b36af5ad99dcab09d30'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.073928 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3d9a5a2c9b4b36af5ad99dcab09d30'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.089218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4bfd599f-5a46-4155-be33-f9126b7745c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3d9a5a2c9b4b36af5ad99dcab09d30'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.095925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1405/2809] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:49 2022] DELETE /identity/v3/users/5c3d9a5a2c9b4b36af5ad99dcab09d30 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:50.423769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:50.425683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:50.445435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99136676-d67e-461e-8b82-b09d12c486b8 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:50.514970 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:50.515657 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.515940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.516212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] PATH_INFO: `/v3/projects/41dc21dba93942768680f772d2b366c9` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.517404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dc21dba93942768680f772d2b366c9)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:50.519377 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.578107 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dc21dba93942768680f772d2b366c9', 'request_id': 'req-311295f3-d30c-48cb-9fef-01ac649f4882'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.585585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dc21dba93942768680f772d2b366c9', 'request_id': 'req-311295f3-d30c-48cb-9fef-01ac649f4882'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.595957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dc21dba93942768680f772d2b366c9', 'request_id': 'req-311295f3-d30c-48cb-9fef-01ac649f4882'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.612043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] The token cache is being invalidate because project 41dc21dba93942768680f772d2b366c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:01:50.612253 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.612880 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-311295f3-d30c-48cb-9fef-01ac649f4882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:50.614413 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1405/2810] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:50 2022] DELETE /identity/v3/projects/41dc21dba93942768680f772d2b366c9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:50.620787 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.621073 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.621352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.643191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:50.644469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:50.711233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1406/2811] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:50.720325 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.720769 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.721105 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.743449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:50.744699 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:50.753920 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1406/2812] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:50.758515 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:50.769256 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e3ac0-98c1-4631-b222-b028dc12855c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:50.774471 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:50.775099 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.775413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.775711 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.776362 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:01:50.776664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:50.778257 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.790777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1407/2813] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:01:50.795697 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:50.805319 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de1f8-c79a-4d38-9f63-40cf46397f80 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:50.810420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:50.811023 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.811330 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.811622 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.812234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:50.814179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.844878 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1407/2814] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:01:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:50.849189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:50.860213 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5930d-1ebd-46e7-b741-e937c82b002f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:50.889006 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:50.925558 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:50.926182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.926461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.926732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.927413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:50.929474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.956817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1408/2815] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:01:50 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:01:50.958211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:50.969321 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aba535-542c-4996-825c-00e84bac0c43 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:50.974336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:50.975001 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:50.975281 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:50.975571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:50.976222 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:50.977838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:50.988756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1408/2816] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:50.993209 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:51.003987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71997f-20c1-4a4a-80db-f6ce96bf93ec admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:51.008946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:51.009964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:51.009964 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:51.010164 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:51.010772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:51.012502 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe807ce-4bc0-4906-9f4d-a874048e1f97 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:51.024411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1409/2817] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:01:51.028963 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:51.039472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fc302-107f-4ae1-b814-173c971fedb6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:51.047079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:51.048040 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:51.048667 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:51.048768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] PATH_INFO: `/v3/projects/91ed638ab95f4f21ad4bcc886adf0df4/users/0236b1a2975247c9b325117f6918c80e/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:51.075202 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ed638ab95f4f21ad4bcc886adf0df4, user_id=0236b1a2975247c9b325117f6918c80e, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:51.077514 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:51.092041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1409/2818] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:01:51 2022] PUT /identity/v3/projects/91ed638ab95f4f21ad4bcc886adf0df4/users/0236b1a2975247c9b325117f6918c80e/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:51.098542 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7754-21b7-45e0-ade4-b8588e7043f7 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:51.099064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7754-21b7-45e0-ade4-b8588e7043f7 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:51.099064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e7754-21b7-45e0-ade4-b8588e7043f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:51.118786 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-6a5e7754-21b7-45e0-ade4-b8588e7043f7 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:51.120032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6a5e7754-21b7-45e0-ade4-b8588e7043f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:51.186224 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1410/2819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:51.196456 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:51.206079 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:51.213458 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:51.270175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a200e6d-43b1-43f0-9893-eb0dddbafb1f admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:51.274083 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:51.274707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:51.274988 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:51.275265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:51.285889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:51.287259 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:51.297370 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1410/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:52.022493 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533d91-f80e-47e3-b736-52615b29e2ad None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:52.022724 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533d91-f80e-47e3-b736-52615b29e2ad None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:52.023047 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d533d91-f80e-47e3-b736-52615b29e2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:52.056348 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-0d533d91-f80e-47e3-b736-52615b29e2ad None None] MFA Rules not processed for user `0236b1a2975247c9b325117f6918c80e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:52.057596 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-0d533d91-f80e-47e3-b736-52615b29e2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:52.144986 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1411/2821] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:52.157778 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:52.159437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:52.225431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2221c-e139-4302-98cf-3ceec595fd53 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:52.229146 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:52.229773 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:52.230048 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:52.230311 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:52.239414 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:52.240898 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:52.250317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1411/2822] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:53.697946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae930d-8a3c-4a86-966f-7f0df014f31d None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:53.698569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae930d-8a3c-4a86-966f-7f0df014f31d None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:53.698569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae930d-8a3c-4a86-966f-7f0df014f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:53.719569 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-dfae930d-8a3c-4a86-966f-7f0df014f31d None None] MFA Rules not processed for user `0236b1a2975247c9b325117f6918c80e`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:53.720812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-dfae930d-8a3c-4a86-966f-7f0df014f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:53.728491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1412/2823] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:01:59.151602 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:59.153663 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:59.185747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a703c-dc53-42e2-b0aa-aa75d13b6845 service cinder] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:59.196021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:59.197616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:59.197845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:59.198175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] PATH_INFO: `/v3/users/0ea811e3d79c4f8aa7beb28aa0230dc5` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:59.200219 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea811e3d79c4f8aa7beb28aa0230dc5)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:59.203993 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:59.287250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea811e3d79c4f8aa7beb28aa0230dc5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:59.288488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea811e3d79c4f8aa7beb28aa0230dc5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:59.292287 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea811e3d79c4f8aa7beb28aa0230dc5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:59.299597 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea811e3d79c4f8aa7beb28aa0230dc5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:59.330937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea811e3d79c4f8aa7beb28aa0230dc5'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:01:59.354157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1412/2824] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:01:59 2022] DELETE /identity/v3/users/0ea811e3d79c4f8aa7beb28aa0230dc5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:01:59.359002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:59.359651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:59.359651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:59.386540 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:01:59.387750 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:59.423511 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:59.476824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1413/2825] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:01:59.489264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:59.491395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:59.535819 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:01:59.577520 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587d44c-d5ba-4bc7-a69f-27fd0dde6f81 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:59.581314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:59.581954 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:59.582231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:59.582502 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:59.593088 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:59.594571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:01:59.605687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1413/2826] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:01:59.931639 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:01:59.933394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:01:59.947490 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cacb5d-e3aa-4e33-ba21-6a0d248fecc0 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:01:59.955132 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:01:59.956000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:01:59.956365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:01:59.956676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] PATH_INFO: `/v3/projects/eed199c167c8478f845f858b9cb736b3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:01:59.959373 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed199c167c8478f845f858b9cb736b3)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:01:59.961427 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:00.012706 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed199c167c8478f845f858b9cb736b3', 'request_id': 'req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.020051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed199c167c8478f845f858b9cb736b3', 'request_id': 'req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.027812 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed199c167c8478f845f858b9cb736b3', 'request_id': 'req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.042138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] The token cache is being invalidate because project eed199c167c8478f845f858b9cb736b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:02:00.042402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.043138 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f326b4b5-b7ca-49f2-ba0e-6eb35e0e5ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.044895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1414/2827] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:01:59 2022] DELETE /identity/v3/projects/eed199c167c8478f845f858b9cb736b3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:00.378437 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:00.380152 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:00.394583 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8b006-238c-4bfd-b7be-3b18a9c90df8 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:00.455767 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:00.456417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:00.456691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:00.456957 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] PATH_INFO: `/v3/users/7fd8b341729b4040813b34104e53e818` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:00.458145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd8b341729b4040813b34104e53e818)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:00.460196 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:00.511405 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd8b341729b4040813b34104e53e818'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.512063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd8b341729b4040813b34104e53e818'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.512649 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd8b341729b4040813b34104e53e818'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.521825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd8b341729b4040813b34104e53e818'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.537779 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd8b341729b4040813b34104e53e818'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:00.552795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1414/2828] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:02:00 2022] DELETE /identity/v3/users/7fd8b341729b4040813b34104e53e818 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:00.557605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:00.558614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:00.558891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:00.580461 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:00.581795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:00.648685 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1415/2829] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:00.659203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:00.660948 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:00.710867 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:00.723669 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c3013-6386-496a-8fb4-29a1fa0894d7 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:00.727691 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:00.728301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:00.728577 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:00.728848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:00.739761 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:00.741242 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e36ad-56c5-473e-9a58-1092d4e28378 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:00.752786 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1415/2830] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:01.059669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.059900 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:01.084258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c9b67-5a34-4723-833a-3361b6b793b8 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.091892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.092553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.092822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.093084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] PATH_INFO: `/v3/projects/50ed76a8467247c4b17af450ebb608fb` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.095839 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ed76a8467247c4b17af450ebb608fb)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.098050 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.168864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ed76a8467247c4b17af450ebb608fb', 'request_id': 'req-4f032781-6839-446f-ae75-9942f04a232c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:01.170834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:01.181325 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ed76a8467247c4b17af450ebb608fb', 'request_id': 'req-4f032781-6839-446f-ae75-9942f04a232c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:01.192199 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ed76a8467247c4b17af450ebb608fb', 'request_id': 'req-4f032781-6839-446f-ae75-9942f04a232c'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:01.212075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] The token cache is being invalidate because project 50ed76a8467247c4b17af450ebb608fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:02:01.212753 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:01.213614 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-4f032781-6839-446f-ae75-9942f04a232c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:01.219352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1416/2831] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:02:01 2022] DELETE /identity/v3/projects/50ed76a8467247c4b17af450ebb608fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:01.221284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.221573 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.221845 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.247201 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:01.249059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:01.326350 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1416/2832] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:01.335382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.335826 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.336202 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.358264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:01.359630 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:01.368155 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1417/2833] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 00:02:01.372848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.383554 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a30bd8-ce45-4d11-ad12-5be751c78b3c None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.390137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.390882 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.391184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.391525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.392203 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:02:01.392508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.394181 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.407441 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1417/2834] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:02:01.412207 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.422000 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a90e12-c2c0-4977-b309-5ab4ee9e655e None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.426695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.427681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.427822 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.428093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.428777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.430616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.461421 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1418/2835] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:02:01 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:01.466895 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.477964 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ccf6b-80d6-46d7-a78c-1129169db42d admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.538435 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.539033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.539372 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.539640 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.540275 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.542209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.569821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1418/2836] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:02:01 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:01.577927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.597134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a966-1186-44ef-81d2-1532fef8f3e1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.602025 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.602644 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.602920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.603190 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.603908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.605538 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.616978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1419/2837] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:01.621175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.632267 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700ec91-44fe-4b76-870f-e0c4cdb83aa3 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.637299 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.637917 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.638191 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.638485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.639508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.641183 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba0ca0-f8fd-4639-a801-dedb8a4c656f admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.653638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1419/2838] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:01.658118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.669315 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e749889-ced1-479f-8c0c-38446d591cdc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.675034 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.676371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.676651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.676925 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] PATH_INFO: `/v3/projects/6de2fa5da2354ba29d3524d06144efd3/users/54357f09d11f48d4bf7b3dd4ba2f9b6a/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.696582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de2fa5da2354ba29d3524d06144efd3, user_id=54357f09d11f48d4bf7b3dd4ba2f9b6a, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.699387 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:01.714862 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1420/2839] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:02:01 2022] PUT /identity/v3/projects/6de2fa5da2354ba29d3524d06144efd3/users/54357f09d11f48d4bf7b3dd4ba2f9b6a/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:01.789031 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.789307 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.789587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.820953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] MFA Rules not processed for user `54357f09d11f48d4bf7b3dd4ba2f9b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:01.822764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:01.867730 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:01.895161 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1420/2840] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:01.906282 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:01.908170 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:01.983860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd816b-db8c-4d53-ac94-feb102611a88 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:01.987667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:01.988283 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:01.988551 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:01.988832 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:01.997743 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:01.999245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeba41-9151-4aba-b3df-b4e1e4407844 service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:02.008512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1421/2841] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:11.891782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=8 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:12.004227 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=59,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:12.253323 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:12.327899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab6a8-c9d3-4de3-b8ac-f8bbecb445d8 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:12.333157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:12.333802 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:12.334075 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:12.334343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] PATH_INFO: `/v3/users/54357f09d11f48d4bf7b3dd4ba2f9b6a` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:12.335536 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] RBAC: Authorizing `identity:delete_user(user_id=54357f09d11f48d4bf7b3dd4ba2f9b6a)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:12.337481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:12.392099 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54357f09d11f48d4bf7b3dd4ba2f9b6a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:12.392748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54357f09d11f48d4bf7b3dd4ba2f9b6a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:12.393336 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54357f09d11f48d4bf7b3dd4ba2f9b6a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:12.404749 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54357f09d11f48d4bf7b3dd4ba2f9b6a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:12.419676 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54357f09d11f48d4bf7b3dd4ba2f9b6a'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:12.434305 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1421/2842] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:02:12 2022] DELETE /identity/v3/users/54357f09d11f48d4bf7b3dd4ba2f9b6a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:12.439084 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:12.439402 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:12.439675 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:12.460605 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:12.462062 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:12.539834 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1422/2843] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:12.549741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:12.551430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:12.612652 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1f54f-c87b-4e19-b31a-121ef09d113b admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:12.616431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:12.617054 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:12.617328 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:12.617596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:12.627343 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:12.628941 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671c3dc-7d8e-43cc-9488-5d318a782a2b service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:12.640530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1422/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:02:12.930485 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:12.932122 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:12.946950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f30b9-f71f-4bff-8e12-073da2d5ba9e None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:12.952250 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:12.953894 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:12.954371 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:12.954643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] PATH_INFO: `/v3/projects/6de2fa5da2354ba29d3524d06144efd3` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:12.955891 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de2fa5da2354ba29d3524d06144efd3)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:12.957843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.010714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de2fa5da2354ba29d3524d06144efd3', 'request_id': 'req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:13.017571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de2fa5da2354ba29d3524d06144efd3', 'request_id': 'req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:13.024068 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de2fa5da2354ba29d3524d06144efd3', 'request_id': 'req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:13.040943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] The token cache is being invalidate because project 6de2fa5da2354ba29d3524d06144efd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:02:13.040943 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:13.041318 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-3544e0fd-ab5c-4eff-a0e1-60caf2b2f72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:13.043268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1423/2845] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:02:12 2022] DELETE /identity/v3/projects/6de2fa5da2354ba29d3524d06144efd3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:13.048541 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.049155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.049155 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.082113 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:13.083455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:13.116928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:13.173368 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1423/2846] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:13.181252 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.181523 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.181795 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.208366 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:13.209581 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:13.218950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1424/2847] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:13.223506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.233486 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8c37e-c16d-4e68-8237-2c58edae2d32 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.238546 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.239147 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.239444 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.239710 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.240355 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:02:13.240666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.242264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.254179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1424/2848] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:02:13.258981 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.269185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5672f-6e67-4759-8bb8-17a70e419107 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.274756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.275390 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.275672 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.275947 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.276579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.278598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.310671 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1425/2849] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:02:13 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:13.315051 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.326263 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39f087-08a1-4d5d-9c3b-a11dfdaaa211 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.387150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.387708 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.387983 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.388284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.389283 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.391209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.415871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1425/2850] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:02:13 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:13.420358 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.431049 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1609a-8af7-48c0-ae15-60f72c652524 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.436507 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.437136 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.437524 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.437652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.438308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.439946 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.451534 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1426/2851] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 00:02:13.455886 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.467063 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4eee4-e706-40e0-87e4-b57674c6f761 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.472838 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.473494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.473826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.474087 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.474742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.476394 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6357d82-51d6-40b3-b49a-e36b7599632d admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.489257 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1426/2852] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:13.493807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.504418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b5dd-8b86-4513-9afd-68376f7ff06f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.510143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.510909 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.511184 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.511627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] PATH_INFO: `/v3/projects/91502d4f18774f6c8d1640893889ec4e/users/c49c3c7e6a454ab5bc692a7a7b8df8cf/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.528860 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91502d4f18774f6c8d1640893889ec4e, user_id=c49c3c7e6a454ab5bc692a7a7b8df8cf, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.531532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.546112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1427/2853] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:02:13 2022] PUT /identity/v3/projects/91502d4f18774f6c8d1640893889ec4e/users/c49c3c7e6a454ab5bc692a7a7b8df8cf/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:13.558645 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba0b9-fa73-4832-ba87-9da6b9ab925a None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.558928 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba0b9-fa73-4832-ba87-9da6b9ab925a None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.559200 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba0b9-fa73-4832-ba87-9da6b9ab925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.589638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-f74ba0b9-fa73-4832-ba87-9da6b9ab925a None None] MFA Rules not processed for user `c49c3c7e6a454ab5bc692a7a7b8df8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:13.591049 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f74ba0b9-fa73-4832-ba87-9da6b9ab925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:13.660675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1427/2854] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:13.672176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:13.674210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:13.697018 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:13.736229 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97ba1-0d0c-4a04-ba7c-2e35937d2c5e admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:13.740022 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d52c53e1244d25b0e16374101ea139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:13.740627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:13.740918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:13.741172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:13.749465 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:13.750884 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44411445-427b-4540-9c81-fff6fff58cff service cinder] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:13.759584 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1428/2855] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:15.177897 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:15.178309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:15.179015 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:15.202910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] MFA Rules not processed for user `c49c3c7e6a454ab5bc692a7a7b8df8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:15.204156 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:15.211553 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1428/2856] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:23.755901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=17 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:25.197333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:29.624615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee46b-4100-449a-9baf-96422ab2761c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:29.625388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee46b-4100-449a-9baf-96422ab2761c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:29.625388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee46b-4100-449a-9baf-96422ab2761c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:29.656752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c96ee46b-4100-449a-9baf-96422ab2761c None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:29.658032 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c96ee46b-4100-449a-9baf-96422ab2761c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:29.724555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1429/2857] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 13 00:02:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:29.777979 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:29.779511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:29.851632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49715a85-28a4-45e7-89e3-9f7714b39aef None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:29.857724 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:29.858361 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:29.858642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:29.858884 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:29.868025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:29.869612 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:29.878571 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1429/2858] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:02:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 00:02:30.412557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d003-7b9e-44ae-b516-01909346bbe4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:30.412880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d003-7b9e-44ae-b516-01909346bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:30.413658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d003-7b9e-44ae-b516-01909346bbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:30.440352 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-5cb5d003-7b9e-44ae-b516-01909346bbe4 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:30.441556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-5cb5d003-7b9e-44ae-b516-01909346bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:30.449233 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1430/2859] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 13 00:02:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:30.461911 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:30.473795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6a080-6596-4f2a-9d0c-f0cacbe0461b service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:30.480240 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:30.480835 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:30.481109 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:30.481373 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:30.489841 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:30.491409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:30.499388 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1430/2860] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:02:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 00:02:30.762571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283518cc-0f8f-4c0e-b548-cdaee6769da8 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:30.763083 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283518cc-0f8f-4c0e-b548-cdaee6769da8 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:30.763185 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283518cc-0f8f-4c0e-b548-cdaee6769da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:30.784513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-283518cc-0f8f-4c0e-b548-cdaee6769da8 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:30.785737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-283518cc-0f8f-4c0e-b548-cdaee6769da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:30.793864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1431/2861] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Sep 13 00:02:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:30.806269 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:30.817407 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaa5a7-e664-423d-ad32-a170578adfd4 service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:30.823211 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:30.823908 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:30.824162 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:30.824409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:30.833619 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:30.835233 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:30.843593 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1431/2862] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:02:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 00:02:31.211301 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:31.211615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:31.212003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:31.235268 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] MFA Rules not processed for user `b20995b417fc43eaac4e709042af9d24`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:31.236556 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:31.245079 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1432/2863] 10.222.0.26 () {58 vars in 967 bytes} [Tue Sep 13 00:02:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:31.258256 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:31.273582 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365369a-5969-4752-aa67-2c172508f95d service swift] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:31.280172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cb125ada8d64bc49ff99383d05aa69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:31.280776 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:31.281154 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:31.281543 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:31.292234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:31.294671 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775612f7-b722-4cab-9a1e-db0c8e48f2b1 service swift] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:31.308041 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1432/2864] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Sep 13 00:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 00:02:33.485800 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:33.554299 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1263082-2d87-4643-ace1-07215a5c63d4 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:33.559821 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:33.560454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:33.560727 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:33.560992 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] PATH_INFO: `/v3/users/c49c3c7e6a454ab5bc692a7a7b8df8cf` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:33.562163 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49c3c7e6a454ab5bc692a7a7b8df8cf)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:33.564203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:33.622381 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49c3c7e6a454ab5bc692a7a7b8df8cf'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:33.623392 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49c3c7e6a454ab5bc692a7a7b8df8cf'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:33.632447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49c3c7e6a454ab5bc692a7a7b8df8cf'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:33.642456 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49c3c7e6a454ab5bc692a7a7b8df8cf'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:33.653667 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:33.655172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49c3c7e6a454ab5bc692a7a7b8df8cf'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:33.662974 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1433/2865] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:02:33 2022] DELETE /identity/v3/users/c49c3c7e6a454ab5bc692a7a7b8df8cf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:33.667662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:33.668170 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:33.668284 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:33.687810 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:33.689157 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:33.758516 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1433/2866] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:33.769258 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:33.770765 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:33.839653 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dff4bb-7c50-4835-8899-2be6769a94d8 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:33.843831 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:33.844431 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:33.844710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:33.844978 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:33.854968 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:33.856443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865afb9-0bfb-4b72-8b21-69116726e7a0 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:33.866230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1434/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:34.051131 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.052626 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:34.065459 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa4a5-8806-4bac-99b4-f7421356c25d None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.072468 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.073010 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.073381 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.073828 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] PATH_INFO: `/v3/projects/91502d4f18774f6c8d1640893889ec4e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.075574 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91502d4f18774f6c8d1640893889ec4e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.078309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.126316 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:34.129282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91502d4f18774f6c8d1640893889ec4e', 'request_id': 'req-79189198-32a8-49d3-9912-0a1fa8d507d9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:34.136035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91502d4f18774f6c8d1640893889ec4e', 'request_id': 'req-79189198-32a8-49d3-9912-0a1fa8d507d9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:34.144411 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91502d4f18774f6c8d1640893889ec4e', 'request_id': 'req-79189198-32a8-49d3-9912-0a1fa8d507d9'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:34.155011 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] The token cache is being invalidate because project 91502d4f18774f6c8d1640893889ec4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:02:34.155334 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:34.155956 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-79189198-32a8-49d3-9912-0a1fa8d507d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:02:34.157461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1434/2868] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:02:34 2022] DELETE /identity/v3/projects/91502d4f18774f6c8d1640893889ec4e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:34.162481 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.162762 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.163030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.182890 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:34.184124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:34.243248 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1435/2869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:34.252436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.252751 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.253029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.274296 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:34.275551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:34.284425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1435/2870] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:34.288902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.299664 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd2063-e266-45c6-afc8-2fc592e327bd None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.306337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.306337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.306337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.306651 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] PATH_INFO: `/v3/domains` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.307071 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87847) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:02:34.307400 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.308987 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.321756 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1436/2871] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:02:34.326241 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.335848 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef839ed0-2e9e-447e-8466-8f737e119ffb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.340735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.341333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.341604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.341869 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.342508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.344454 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.375045 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1436/2872] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:02:34 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:34.379520 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.390553 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2abe09-1c64-47ba-b4c3-ade4fa641001 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.449312 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.449924 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.450206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.450467 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.451106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.453056 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.477544 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1437/2873] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:02:34 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:02:34.483961 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.499799 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40081ade-5bb9-4e07-ad9b-ad28160831a2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.505605 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.506204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.506481 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.506752 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.507436 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.509056 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.519736 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1437/2874] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:34.524075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.539116 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0db787-93f2-4375-9f8a-7d917785f2bc admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.546354 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.547206 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.547629 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.547913 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.548757 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.550346 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105264c8-ef8d-4394-9441-54699078b086 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.561624 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1438/2875] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:34.565918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.576172 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151cc2-f2ce-42ab-be16-96761625a745 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.581250 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.581900 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.582177 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.582455 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] PATH_INFO: `/v3/projects/ecac3196aefa4c32994f85a86849c8c8/users/f72b7cf8bf38477d8d6534eda6611982/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.597551 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecac3196aefa4c32994f85a86849c8c8, user_id=f72b7cf8bf38477d8d6534eda6611982, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.600160 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.614875 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1438/2876] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:02:34 2022] PUT /identity/v3/projects/ecac3196aefa4c32994f85a86849c8c8/users/f72b7cf8bf38477d8d6534eda6611982/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:34.620137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c08ed-3f0c-46da-ba93-6cad6597a36c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.620425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c08ed-3f0c-46da-ba93-6cad6597a36c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.620690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c08ed-3f0c-46da-ba93-6cad6597a36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.628879 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:34.641961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-194c08ed-3f0c-46da-ba93-6cad6597a36c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:34.643322 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-194c08ed-3f0c-46da-ba93-6cad6597a36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:34.706754 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1439/2877] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:34.716901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:34.718763 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:34.778273 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bc725-d937-4d7b-98c3-c0640129c352 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:34.782315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:34.782946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:34.783213 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:34.783513 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:34.792985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:34.794422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:34.804533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1439/2878] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:38.893433 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:38.894411 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:38.894697 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:38.926378 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] MFA Rules not processed for user `f72b7cf8bf38477d8d6534eda6611982`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:38.927773 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:38.998270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1440/2879] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:39.003006 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.013430 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6524571-4c12-4d72-a516-64952e5fc034 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.018824 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.019417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.019686 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.019960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] PATH_INFO: `/v3/projects` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.020604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.022565 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.051566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1440/2880] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:02:39 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:02:39.055852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.074063 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf861c8c-6c16-407a-a681-8a687a356764 None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.135130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.135828 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.136106 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.136368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] PATH_INFO: `/v3/users` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.137017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.139494 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.163192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1441/2881] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:02:39 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:02:39.167315 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.178195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f3a0b-af59-4b02-877a-6cb027cc0633 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.183243 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.183855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.184126 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.184393 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.185039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.186648 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.202881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1441/2882] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:39.204196 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.215307 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b797e-1a62-432b-a6e3-bd8257f9ff55 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.220833 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.221525 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.221824 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.222104 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] PATH_INFO: `/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8/users/cca074ca906340a49a731ad9b6f13042/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.239996 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73ae26a150d4fc9b18752d0ee711ff8, user_id=cca074ca906340a49a731ad9b6f13042, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.242920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.253037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:39.258264 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1442/2883] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:02:39 2022] PUT /identity/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8/users/cca074ca906340a49a731ad9b6f13042/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:39.262557 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.273144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91def85e-eddb-4cd9-9deb-833f4e1ed0a2 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.312188 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:39.335293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.336007 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.336422 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.336692 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.337404 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.339028 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07961bc5-c4c2-415f-a338-83763a9579ed admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.352446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1442/2884] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:02:39.356892 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:39.368057 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490895d6-9da5-4c1e-8899-fd0ae6e4a2da admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:39.373907 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:39.374575 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:39.374868 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:39.375130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] PATH_INFO: `/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8/users/cca074ca906340a49a731ad9b6f13042/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:39.377183 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73ae26a150d4fc9b18752d0ee711ff8, user_id=cca074ca906340a49a731ad9b6f13042, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:39.379864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:39.391464 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1443/2885] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:02:39 2022] PUT /identity/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8/users/cca074ca906340a49a731ad9b6f13042/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:02:44.218764 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe338-79e9-4b95-974b-231216259a20 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:44.220432 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe338-79e9-4b95-974b-231216259a20 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:44.223815 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fe338-79e9-4b95-974b-231216259a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:44.288491 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d65fe338-79e9-4b95-974b-231216259a20 None None] MFA Rules not processed for user `cca074ca906340a49a731ad9b6f13042`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:02:44.290143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d65fe338-79e9-4b95-974b-231216259a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:44.385465 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1443/2886] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:02:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:02:44.405758 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:02:44.408811 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:02:44.507365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b917-0a4e-4637-8345-497214d3fcb4 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:02:44.512966 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244d121-256c-452e-9104-1e608a29599a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:02:44.513695 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d121-256c-452e-9104-1e608a29599a service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:02:44.514342 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d121-256c-452e-9104-1e608a29599a service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:02:44.514621 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d121-256c-452e-9104-1e608a29599a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:02:44.581305 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244d121-256c-452e-9104-1e608a29599a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:02:44.583200 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244d121-256c-452e-9104-1e608a29599a service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:02:44.594447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1444/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:02:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:02:54.381689 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=45 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:02:54.588747 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:00.242322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:00.242892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:00.242892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:00.265061 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] MFA Rules not processed for user `0236b1a2975247c9b325117f6918c80e`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:03:00.267433 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:00.335833 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1444/2888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:03:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:03:00.376051 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244d121-256c-452e-9104-1e608a29599a service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:00.377737 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2244d121-256c-452e-9104-1e608a29599a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:00.438541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244d121-256c-452e-9104-1e608a29599a service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:00.443512 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afd5af60bde74ab0a3e8f3f2d17ed8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:00.444140 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:00.444413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:00.444681 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:00.446683 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:00.462274 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:00.463799 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c5afd-4d63-46d4-8a6b-5116731a0e77 service glance] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:00.475643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1445/2889] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:03:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:03:01.030976 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:01.094821 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c614-8be4-419e-be3d-dfe8553f8643 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:01.099282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:01.099930 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:01.100344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:01.100624 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:01.102504 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:01.116950 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:01.118379 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:01.130483 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1445/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:03:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:03:04.247980 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:04.248557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:04.248557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:04.269154 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] MFA Rules not processed for user `f72b7cf8bf38477d8d6534eda6611982`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:03:04.270384 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:04.278923 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1446/2891] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:03:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:03:11.125575 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=51 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:14.262918 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=29 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:27.364766 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:27.377314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a7ce-691c-49f2-8cad-1613d3a3ccb2 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:27.382013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:27.382632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:27.383039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:27.383345 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:27.395390 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:27.396826 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:27.406966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1446/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:03:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:03:37.402231 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:56.838455 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:56.840093 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:56.897788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:56.899852 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:56.916476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a8c83-b62f-453f-96aa-54fecaba6dac None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:56.921526 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:56.922141 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:56.922416 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:56.922678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:56.924415 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:56.936027 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:56.937616 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:56.948198 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1447/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:03:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:03:56.972921 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b22ee-7dbb-4052-9c47-bf9fc3475c83 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:56.978754 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:56.979494 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:56.979871 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:56.980030 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] PATH_INFO: `/v3/users/0236b1a2975247c9b325117f6918c80e` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:56.981300 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0236b1a2975247c9b325117f6918c80e)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:56.983282 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.033002 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0236b1a2975247c9b325117f6918c80e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.033782 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0236b1a2975247c9b325117f6918c80e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.034371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0236b1a2975247c9b325117f6918c80e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.041585 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0236b1a2975247c9b325117f6918c80e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.057748 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-73ac5709-9446-4d4e-9b79-495b3d6b1615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0236b1a2975247c9b325117f6918c80e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.079425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1447/2894] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:03:56 2022] DELETE /identity/v3/users/0236b1a2975247c9b325117f6918c80e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:03:57.264802 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.281587 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e19b-2580-4455-ae46-4ab690f046c4 service placement] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.343504 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.344114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.344394 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.344668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] PATH_INFO: `/v3/projects/91ed638ab95f4f21ad4bcc886adf0df4` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.345827 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ed638ab95f4f21ad4bcc886adf0df4)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.347775 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.395326 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ed638ab95f4f21ad4bcc886adf0df4', 'request_id': 'req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.403410 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ed638ab95f4f21ad4bcc886adf0df4', 'request_id': 'req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.410337 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ed638ab95f4f21ad4bcc886adf0df4', 'request_id': 'req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.425951 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] The token cache is being invalidate because project 91ed638ab95f4f21ad4bcc886adf0df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87847) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:03:57.426339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.427189 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f1a3c05f-b1cd-490d-80b3-2c6eb232d2d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:03:57.428843 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1448/2895] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:03:57 2022] DELETE /identity/v3/projects/91ed638ab95f4f21ad4bcc886adf0df4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:03:57.435278 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.435637 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.436003 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.465714 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:03:57.467385 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:57.536636 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1448/2896] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:03:57.549886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.549886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.549886 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.567592 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:03:57.568853 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:57.581477 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1449/2897] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:03:57.586198 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.596864 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51b740-d221-4745-bdae-d1fb049561d4 None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.602014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.602628 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.602913 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.603199 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] PATH_INFO: `/v3/domains` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.603889 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87846) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 00:03:57.604216 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.605846 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.618861 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1449/2898] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Sep 13 00:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 00:03:57.623722 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.633732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213b41-616e-4550-bcc9-1dd300c6445c None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.640288 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.640993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.641210 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.641474 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.642660 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.644443 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.658127 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:57.690368 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1450/2899] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:03:57 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:03:57.694893 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.705472 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630b7ca-b837-4353-9706-86c3f0ea53ed admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.709881 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:03:57.772866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.773638 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.774029 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.774298 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.774926 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.776899 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.807866 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1450/2900] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:03:57 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:03:57.813541 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.824388 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262f6b7-5573-4f30-962c-c3534b3bbd6f admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.829124 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.829736 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.830008 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.830276 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.830911 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.832579 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.843550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1451/2901] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:03:57.847771 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.858679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5a75-8331-48a8-a720-9e02d70e2a18 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.864247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.864849 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.865123 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.865395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] PATH_INFO: `/v3/roles` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.866039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.867698 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a59b9-befc-4b2c-a718-d27b5c92d8da admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.878059 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1451/2902] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:03:57.883555 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:57.899179 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee463678-fe9b-4cab-af4e-e74df00342ac admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:57.905510 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:57.906178 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] REQUEST_METHOD: `PUT` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.906454 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.906723 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] PATH_INFO: `/v3/projects/07b62652571943b58f9c8bd4c1ee9931/users/fb103e002840445db31703e141cf3a66/roles/cc467837f6074b9391a11b0e57406d12` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.923075 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b62652571943b58f9c8bd4c1ee9931, user_id=fb103e002840445db31703e141cf3a66, role_id=cc467837f6074b9391a11b0e57406d12)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:57.925797 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:57.940261 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1452/2903] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:03:57 2022] PUT /identity/v3/projects/07b62652571943b58f9c8bd4c1ee9931/users/fb103e002840445db31703e141cf3a66/roles/cc467837f6074b9391a11b0e57406d12 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:03:57.945855 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2e1b8-7032-4a48-8e25-528ac8a52399 None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:57.946366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2e1b8-7032-4a48-8e25-528ac8a52399 None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:57.946496 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2e1b8-7032-4a48-8e25-528ac8a52399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:57.966169 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-2bf2e1b8-7032-4a48-8e25-528ac8a52399 None None] MFA Rules not processed for user `f9f809fdc6b04f0ca4d4c2bc46e0a712`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:03:57.967409 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-2bf2e1b8-7032-4a48-8e25-528ac8a52399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:58.027508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1452/2904] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:03:58.037559 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:03:58.039669 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:03:58.103033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701922e-2846-457d-bb8d-46dfd3c27785 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:03:58.106944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:03:58.107586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:03:58.107869 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:03:58.108142 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:03:58.117678 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:03:58.119108 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:03:58.129492 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1453/2905] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:02.330621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193556-f744-45f6-9624-0c6f047098fb None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.331533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193556-f744-45f6-9624-0c6f047098fb None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.331533 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193556-f744-45f6-9624-0c6f047098fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.366195 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-29193556-f744-45f6-9624-0c6f047098fb None None] MFA Rules not processed for user `fb103e002840445db31703e141cf3a66`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:04:02.367847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-29193556-f744-45f6-9624-0c6f047098fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:02.469274 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1453/2906] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:04:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:04:02.474236 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.487412 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bcaab-2ff9-4037-91da-6404cfcd504e service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.495807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.496491 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.496781 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.497048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] PATH_INFO: `/v3/projects` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.497729 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.500030 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.540901 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1454/2907] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Sep 13 00:04:02 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 00:04:02.541609 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193556-f744-45f6-9624-0c6f047098fb None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.557415 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193556-f744-45f6-9624-0c6f047098fb None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.578247 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:02.640386 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.641021 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.641285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.641552 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] PATH_INFO: `/v3/users` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.642207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.644234 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.671277 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1454/2908] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Sep 13 00:04:02 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 00:04:02.675571 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.688191 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd32f5-50dc-4213-be6e-a3dbad78c3fe admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.694536 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.695157 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.695469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.695741 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.696446 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.698176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.710058 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1455/2909] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:04:02.714641 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.725292 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dacb33-9f04-467a-8fe5-2411095745aa admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.730366 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.731024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.731322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.731596 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] PATH_INFO: `/v3/projects/f358fb96b8594cd086421aa80e194d06/users/a43985a9e5bb4475acf49744d038d503/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.747264 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f358fb96b8594cd086421aa80e194d06, user_id=a43985a9e5bb4475acf49744d038d503, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.749984 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.767331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1455/2910] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:04:02 2022] PUT /identity/v3/projects/f358fb96b8594cd086421aa80e194d06/users/a43985a9e5bb4475acf49744d038d503/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:04:02.771658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.781952 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41aa15-c5c3-48d7-b943-fb3577b50cbb admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.843699 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.844218 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.844488 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.844731 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.845612 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.847277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589ad39-8e21-4aab-b1e3-11fc072adc80 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.863286 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1456/2911] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 13 00:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 00:04:02.868417 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:02.884314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8b8a5-7989-42f8-8030-a3d6157a50b4 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:02.892452 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:02.893451 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] REQUEST_METHOD: `PUT` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:02.893788 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:02.894057 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] PATH_INFO: `/v3/projects/f358fb96b8594cd086421aa80e194d06/users/a43985a9e5bb4475acf49744d038d503/roles/c8bd8a06edea4fb7962ff7412be096ff` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:02.896230 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f358fb96b8594cd086421aa80e194d06, user_id=a43985a9e5bb4475acf49744d038d503, role_id=c8bd8a06edea4fb7962ff7412be096ff)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:02.899062 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:02.911342 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1456/2912] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Sep 13 00:04:02 2022] PUT /identity/v3/projects/f358fb96b8594cd086421aa80e194d06/users/a43985a9e5bb4475acf49744d038d503/roles/c8bd8a06edea4fb7962ff7412be096ff => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:04:07.596945 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1a11-5d72-4dff-b9a5-5dfdcfc2df90 None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:07.597658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1a11-5d72-4dff-b9a5-5dfdcfc2df90 None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:07.597658 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1a11-5d72-4dff-b9a5-5dfdcfc2df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:07.633425 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:07.637024 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-73ab1a11-5d72-4dff-b9a5-5dfdcfc2df90 None None] MFA Rules not processed for user `a43985a9e5bb4475acf49744d038d503`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:04:07.640095 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-73ab1a11-5d72-4dff-b9a5-5dfdcfc2df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:07.742611 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1457/2913] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:04:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:04:07.761747 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:07.766986 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:07.866816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736c7b9-eef6-4c84-b378-669b54d9ea87 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:07.875301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:07.876115 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:07.876431 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:07.876642 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:07.946014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:07.947722 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:07.957623 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1457/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:09.037002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:09.037550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:09.037550 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:09.058065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] MFA Rules not processed for user `fb103e002840445db31703e141cf3a66`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:04:09.059418 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:09.067615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1458/2915] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:04:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:04:15.864604 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:15.870478 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:15.931858 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62568791-c304-4491-a1e9-c9b169d63a94 service nova] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:15.935839 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:15.936443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:15.936795 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:15.937069 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:15.938992 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:15.949065 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:15.950499 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:15.959137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1458/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:19.051967 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=31 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:25.773007 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:25.785598 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72808484-cf88-4e64-a263-2c5ecd8ee5be None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:25.790003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:25.790649 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:25.790950 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:25.791235 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:25.793332 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:25.803341 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:25.804807 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:25.813064 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1459/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:25.955425 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=24 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:26.304331 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:26.305796 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:26.379194 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2b0f3-7539-48ad-b1a6-471996e9fe3e service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:26.385125 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:26.385742 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:26.386009 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:26.386285 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:26.388207 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:26.401081 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:26.402679 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:26.413309 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1459/2918] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:29.142792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:29.144615 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:29.162604 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6887-1666-471a-bb5e-6616bbcc730f service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:29.170290 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:29.170914 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:29.171160 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:29.171482 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:29.173582 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:29.187082 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:29.188999 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:29.201182 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1460/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:36.408793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=32 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:37.948587 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:37.964133 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6448ca-01d7-486e-a941-d15dba7d89b7 service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:37.970297 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:37.970903 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:37.971244 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:37.971568 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:37.973337 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:04:37.986301 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:37.987953 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ef5e0-0ec3-4c77-8ab7-ee07e0b6a4b3 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:37.999449 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1460/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:39.196089 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=12 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:41.008676 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:04:41.024897 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7af68b-d193-4e91-b7da-d3b178bfc1c4 service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:04:41.031028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:04:41.031662 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:04:41.031936 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:04:41.032201 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:04:41.043830 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:04:41.045296 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:04:41.056451 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1461/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:04:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:04:47.994621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:04:51.051121 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:05:32.592036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] REQUEST_METHOD: `POST` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:05:32.592687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:05:32.592687 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:05:32.614966 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.auth.core [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] MFA Rules not processed for user `f72b7cf8bf38477d8d6534eda6611982`. Rule list: `[]` (Enabled: `True`). {{(pid=87846) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:05:32.616258 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:05:32.676265 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1461/2922] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:05:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:05:33.048829 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:05:33.058610 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df094a8b-d860-4189-b829-1fadbe1449d5 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:05:33.062920 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:05:33.063593 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:05:33.063836 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:05:33.064137 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:05:33.066074 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:05:33.076710 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:05:33.078176 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:05:33.087267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1462/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:05:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:05:42.668960 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=25 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:05:43.083076 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:06:14.185530 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:06:14.198876 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6accd22-afe5-40df-a70c-0719157bb29e None None] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:06:14.205004 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:06:14.205632 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:06:14.206035 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:06:14.206209 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:06:14.208151 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:06:14.220091 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:06:14.221670 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:06:14.232039 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1462/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:06:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:06:24.227614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:06:56.409532 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:06:56.411123 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:06:56.422370 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aae0e4-5037-431c-8ddf-f551a7938856 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:06:56.426837 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:06:56.427447 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:06:56.427734 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:06:56.428002 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:06:56.429948 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:06:56.440365 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:06:56.441792 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a17f9a-7b55-47df-bbb8-4cadda630a7b service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:06:56.451003 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1463/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:06:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:06:56.534825 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:06:56.545116 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f923b6b-6645-47b7-8ba1-5ac46f768ebe service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:06:56.548937 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:06:56.549535 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:06:56.549811 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:06:56.550080 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:06:56.551904 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:06:56.565580 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:06:56.567022 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:06:56.578333 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1463/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:06:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:07:06.447308 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=6 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:07:06.573506 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:07:43.981864 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] REQUEST_METHOD: `POST` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:07:43.982432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:07:43.982432 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:07:44.002915 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.auth.core [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] MFA Rules not processed for user `f72b7cf8bf38477d8d6534eda6611982`. Rule list: `[]` (Enabled: `True`). {{(pid=87847) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 00:07:44.004134 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:07:44.012017 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1464/2927] 10.222.0.26 () {58 vars in 958 bytes} [Tue Sep 13 00:07:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 00:07:44.105741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:07:44.116150 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b652-9ef8-4289-a612-eabb28a73179 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:07:44.120488 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:07:44.121097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:07:44.121395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:07:44.121662 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:07:44.124000 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:07:44.136712 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:07:44.138511 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:07:44.147635 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1464/2928] 10.222.0.26 () {60 vars in 1396 bytes} [Tue Sep 13 00:07:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:07:53.997420 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=3 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:07:54.143566 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:01.543469 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:01.559908 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e969c5-3990-4c96-b31f-5164bb21757f None None] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:01.564665 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:01.565272 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:01.565543 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:01.565810 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:01.567880 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:01.583625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:01.585070 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:01.598001 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1465/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:08:01.789317 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:01.799428 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e06770-1116-4dc8-ad99-cfabee465248 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:01.804033 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:01.804614 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:01.804857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:01.805145 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:01.807420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:01.821746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:01.823166 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:01.834357 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1465/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:08:03.625172 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:03.636772 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23253-9388-4fe2-a4b2-5217b8225ff1 service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:03.643130 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:03.643774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:03.644052 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:03.644324 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:03.646118 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:03.658617 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:03.660230 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:03.671476 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1466/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:08:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:08:11.829647 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=6 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:13.666486 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=12 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:33.890946 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:33.902746 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d736e-efe4-4a70-9962-a9a3c0465ff9 service neutron] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:33.908696 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:33.909344 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:33.909639 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:33.909910 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:33.911896 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:33.925164 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:33.926816 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:33.943476 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1466/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:08:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:08:37.101819 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:37.167042 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7de9-7fe2-4cd4-8de3-3b1535369c2f service nova] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:37.171732 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:37.172351 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:37.172625 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:37.172895 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] PATH_INFO: `/v3/users/fb103e002840445db31703e141cf3a66` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:37.174060 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb103e002840445db31703e141cf3a66)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:37.175961 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:37.226289 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb103e002840445db31703e141cf3a66'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.226944 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb103e002840445db31703e141cf3a66'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.234192 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb103e002840445db31703e141cf3a66'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.244463 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb103e002840445db31703e141cf3a66'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.257143 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb103e002840445db31703e141cf3a66'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.263499 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1467/2933] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:08:37 2022] DELETE /identity/v3/users/fb103e002840445db31703e141cf3a66 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:08:37.456055 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:37.468985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49dc51-d72b-4c5a-a7bf-e4ea9d1a86c0 service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:37.530512 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:37.531175 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:37.531482 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:37.531756 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] PATH_INFO: `/v3/projects/07b62652571943b58f9c8bd4c1ee9931` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:37.532918 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b62652571943b58f9c8bd4c1ee9931)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:37.534842 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:37.584728 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b62652571943b58f9c8bd4c1ee9931', 'request_id': 'req-e9dccb5a-9d44-4dfe-8461-c849df0d359e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.594137 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b62652571943b58f9c8bd4c1ee9931', 'request_id': 'req-e9dccb5a-9d44-4dfe-8461-c849df0d359e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.606892 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b62652571943b58f9c8bd4c1ee9931', 'request_id': 'req-e9dccb5a-9d44-4dfe-8461-c849df0d359e'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.619854 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] The token cache is being invalidate because project 07b62652571943b58f9c8bd4c1ee9931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:08:37.620187 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.621071 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.623266 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1467/2934] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:08:37 2022] DELETE /identity/v3/projects/07b62652571943b58f9c8bd4c1ee9931 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:08:37.628993 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:37.630898 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:37.645430 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8d452-878d-484f-8ff3-00e00470e6e7 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:37.713852 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:37.714489 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:37.714769 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:37.715043 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] PATH_INFO: `/v3/users/a43985a9e5bb4475acf49744d038d503` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:37.716382 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43985a9e5bb4475acf49744d038d503)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:37.718355 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:37.743927 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:37.770643 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43985a9e5bb4475acf49744d038d503'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.771262 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43985a9e5bb4475acf49744d038d503'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.779162 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43985a9e5bb4475acf49744d038d503'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.788962 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43985a9e5bb4475acf49744d038d503'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.801935 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43985a9e5bb4475acf49744d038d503'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:37.808010 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1468/2935] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:08:37 2022] DELETE /identity/v3/users/a43985a9e5bb4475acf49744d038d503 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:08:38.008675 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:08:38.010340 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:08:38.028013 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dccb5a-9d44-4dfe-8461-c849df0d359e admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:08:38.097249 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:08:38.097901 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:08:38.098204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:08:38.098479 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] PATH_INFO: `/v3/projects/f358fb96b8594cd086421aa80e194d06` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:08:38.099831 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f358fb96b8594cd086421aa80e194d06)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:08:38.101793 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:08:38.127616 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:38.158713 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f358fb96b8594cd086421aa80e194d06', 'request_id': 'req-db1739c0-d1dd-4703-a7af-db4d78d775c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:38.165290 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f358fb96b8594cd086421aa80e194d06', 'request_id': 'req-db1739c0-d1dd-4703-a7af-db4d78d775c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:38.173847 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f358fb96b8594cd086421aa80e194d06', 'request_id': 'req-db1739c0-d1dd-4703-a7af-db4d78d775c6'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:38.185024 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] The token cache is being invalidate because project f358fb96b8594cd086421aa80e194d06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:08:38.185322 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:38.186097 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:08:38.187683 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1468/2936] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:08:38 2022] DELETE /identity/v3/projects/f358fb96b8594cd086421aa80e194d06 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:08:47.804622 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:08:48.181655 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:09:20.869774 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:20.871633 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:20.932065 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bd046-2294-466c-b54d-5148283d60d0 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:20.984806 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:20.985413 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:20.985690 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:20.985959 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:20.988080 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:20.999088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:21.001203 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:21.010627 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1469/2937] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:09:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:09:31.007114 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=46 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:09:32.026254 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:32.027931 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:32.131143 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1739c0-d1dd-4703-a7af-db4d78d775c6 admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:32.137860 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97e8cd3ae37443acbb2c7c710646686a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:32.139070 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:32.139598 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:32.140189 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:32.143707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:32.162630 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:32.164768 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:32.181508 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1469/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:09:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:09:42.175144 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=34 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:09:45.016115 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:45.017652 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:45.028428 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6129791-9f2f-4a3b-a58a-01d8fbf739ff service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:45.032436 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c49b9be98a4a9194239c5be186e1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:45.033173 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:45.033339 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:45.033557 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:45.035267 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:45.100599 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:45.102037 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:45.112270 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1470/2939] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:09:51.530461 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:51.532531 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:51.599314 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:51.600752 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:51.615994 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57b6e7-7674-47aa-ba2f-a2a61700861f service neutron] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:51.620684 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:51.621321 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:51.621601 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:51.621902 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] PATH_INFO: `/v3/users/f72b7cf8bf38477d8d6534eda6611982` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:51.623028 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72b7cf8bf38477d8d6534eda6611982)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:51.625088 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:51.632362 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97486929-bffe-4d97-9f0f-31953d292e15 service nova] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:51.638043 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95184693-a582-48ad-b107-277d16240a2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:51.638666 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95184693-a582-48ad-b107-277d16240a2c service placement] REQUEST_METHOD: `GET` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:51.638939 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95184693-a582-48ad-b107-277d16240a2c service placement] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:51.639443 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95184693-a582-48ad-b107-277d16240a2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:51.641205 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-95184693-a582-48ad-b107-277d16240a2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:51.653999 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95184693-a582-48ad-b107-277d16240a2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:51.655843 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95184693-a582-48ad-b107-277d16240a2c service placement] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:51.666894 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1470/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:09:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 00:09:51.683817 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72b7cf8bf38477d8d6534eda6611982'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:51.684498 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72b7cf8bf38477d8d6534eda6611982'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:51.692245 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72b7cf8bf38477d8d6534eda6611982'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:51.700513 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72b7cf8bf38477d8d6534eda6611982'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:51.713033 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72b7cf8bf38477d8d6534eda6611982'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:51.719277 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1471/2941] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:09:51 2022] DELETE /identity/v3/users/f72b7cf8bf38477d8d6534eda6611982 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:09:51.902204 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95184693-a582-48ad-b107-277d16240a2c service placement] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:51.915025 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95184693-a582-48ad-b107-277d16240a2c service placement] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:51.981036 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:51.981656 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:51.981940 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:51.982212 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] PATH_INFO: `/v3/projects/ecac3196aefa4c32994f85a86849c8c8` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:51.983406 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecac3196aefa4c32994f85a86849c8c8)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:51.985314 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:52.037834 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecac3196aefa4c32994f85a86849c8c8', 'request_id': 'req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.044398 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecac3196aefa4c32994f85a86849c8c8', 'request_id': 'req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.054509 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecac3196aefa4c32994f85a86849c8c8', 'request_id': 'req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.069487 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] The token cache is being invalidate because project ecac3196aefa4c32994f85a86849c8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:09:52.069741 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.070395 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.071985 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1471/2942] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:09:51 2022] DELETE /identity/v3/projects/ecac3196aefa4c32994f85a86849c8c8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:09:52.076712 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:52.079749 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:52.097714 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df11b4c-b7f5-4120-89df-69913b85e898 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:52.159777 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:52.160414 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:52.160694 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:52.160965 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] PATH_INFO: `/v3/users/cca074ca906340a49a731ad9b6f13042` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:52.162151 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca074ca906340a49a731ad9b6f13042)` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:52.164112 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:52.169565 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:09:52.212977 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca074ca906340a49a731ad9b6f13042'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.213613 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca074ca906340a49a731ad9b6f13042'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.220692 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca074ca906340a49a731ad9b6f13042'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.228004 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca074ca906340a49a731ad9b6f13042'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.241048 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.notifications [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca074ca906340a49a731ad9b6f13042'} {{(pid=87847) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.246985 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1472/2943] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 13 00:09:52 2022] DELETE /identity/v3/users/cca074ca906340a49a731ad9b6f13042 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 00:09:52.452184 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Authenticating user token {{(pid=87846) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:09:52.453753 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.fernet_utils [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87846) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:09:52.467446 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf6dc4-0fbb-41d6-852c-d08f12eba2fb admin admin] Validating token access rules against request {{(pid=87846) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:09:52.499420 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:09:52.544066 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f809fdc6b04f0ca4d4c2bc46e0a712', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16141215d11e48819148a2d135301f22', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87846) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:09:52.544735 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:09:52.545014 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] SCRIPT_NAME: `/identity` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:09:52.546371 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] PATH_INFO: `/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8` {{(pid=87846) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:09:52.547770 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73ae26a150d4fc9b18752d0ee711ff8)` {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:09:52.549707 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] RBAC: Authorization granted {{(pid=87846) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:09:52.598525 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73ae26a150d4fc9b18752d0ee711ff8', 'request_id': 'req-60b6762b-79bc-458b-99fd-1e2ff0fc8365'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.604857 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73ae26a150d4fc9b18752d0ee711ff8', 'request_id': 'req-60b6762b-79bc-458b-99fd-1e2ff0fc8365'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.612485 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73ae26a150d4fc9b18752d0ee711ff8', 'request_id': 'req-60b6762b-79bc-458b-99fd-1e2ff0fc8365'} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.623179 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] The token cache is being invalidate because project d73ae26a150d4fc9b18752d0ee711ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87846) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 00:09:52.623621 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.624293 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG keystone.notifications [None req-60b6762b-79bc-458b-99fd-1e2ff0fc8365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87846) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 00:09:52.625863 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: [pid: 87846|app: 0|req: 1472/2944] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 13 00:09:52 2022] DELETE /identity/v3/projects/d73ae26a150d4fc9b18752d0ee711ff8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 00:10:02.243586 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:10:02.620220 dv-cinder-820027-26-iscsi devstack@keystone.service[87846]: DEBUG dbcounter [-] [87846] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87846) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 00:11:20.141991 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Authenticating user token {{(pid=87847) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 00:11:20.144061 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:11:20.310931 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd0d55-018d-4d76-9324-0b7e61d666d1 admin admin] Validating token access rules against request {{(pid=87847) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 00:11:20.372459 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '734ed3e452894cbbbe292b78390dc2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4177ed41ebc442889c73d90f9c38b9e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87847) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 00:11:20.373077 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] REQUEST_METHOD: `GET` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 00:11:20.373356 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] SCRIPT_NAME: `/identity` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 00:11:20.373623 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87847) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 00:11:20.375702 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.fernet_utils [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87847) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 00:11:20.388668 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 00:11:20.390186 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f6c8e-77b1-4dec-94cc-47b78db6c414 service placement] RBAC: Authorization granted {{(pid=87847) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 00:11:20.400708 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: [pid: 87847|app: 0|req: 1473/2945] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Sep 13 00:11:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 00:11:30.396444 dv-cinder-820027-26-iscsi devstack@keystone.service[87847]: DEBUG dbcounter [-] [87847] Writing DB stats keystone:SELECT=89 {{(pid=87847) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}